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FOREWORD 


The  Systems  Hanning  Technical  Area  of  the  Army  Research  Institute  (ARI)  is 
concerned  with  improving  the  relationship  between  soldiers  and  the  equipment 
they  use.  This  report  provides  an  evaluation  of  one  particular  set  of  proce¬ 
dures,  the  HARDMAN  (Hardware  vs.  Manpower)  Comparability  Methodology  (HCM) , 
which  is  used  by  the  Navy  to  estimate  manpower,  personnel,  and  training 
requirements  for  proposed  systems  during  the  very  early  stages  of  system  design 
and  development. 

Like  the  Navy,  the  Army  has  been  observing  problems  with  the  mismatch  be¬ 
tween  advancing  technology  and  decreasing  availability  of  soldier  skills.  In 
1960,  ARI  initiated  an  effort,  in  support  of  the  Army-wide  interest  in  the 
system  manning  problem,  to  establish  whether  the  Navy's  HARDMAN  methodology 
could  be  tailored  for  use  on  new  Army  Acquisitions.  Recognizing  that  approxi¬ 
mately  seventy-five  percent  of  the  system  acquisition  cost  is  committed  by 
Phase  II  of  the  development  cycle,  the  Army  was  also  interested  in  determining 
whether  the  technique  could  provide  early,  reasonably  accurate,  manpower,  per¬ 
sonnel,  and  training  (MPT)  estimates.  Because  it  appeared  that  the  HCM  was  the 
most  encompassing  and  mature  of  the  available  MPT  models,  ARI  decided  to  per¬ 
form  test  applications  of  this  technique  as  well  as  a  validation  of  the  method¬ 
ology.  To  date,  the  Army  has  funded  four  test  applications:  Corps  Support 
Weapon  System  (CSWS),  Division  Support  Weapon  System  (DSWS),  Remotely  Piloted 
Vehicle  (RPV),  and  Single  Channel  Ground  Airborne  Radio  System  (SINCGARS). 

The  evaluation  described  in  this  document  represents  the  first  segment  of 
the  Army's  validation  effort  and  draws  on  the  results  of  three  of  the  test 
applications.  The  HARDMAN  methodology  has  good  near-term  utility  as  a  solution 
to  the  Army's  MPT  dilemma. 


EDGAR  M.  JOHNSOIT 
Technical  Director 
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EVALUATION  OF  THE  HARDMAN  (HARDWARE  VS.  MANPOWER)  COMPARABILITY 
METHODOLOGY 


EXECUTIVE  SUMMARY 


Requirement : 

To  improve  the  Army's  capability  to  project  manpower,  personnel,  and 
training  (MPT)  requirements  for  proposed  weapon  systems  (NEW)  by  evaluating 
che  credibility  and  utility  of  the  HARDMAN  (Hardware  vs.  Manpower)  Compara¬ 
bility  Methodology  (HCM),  a  Navy  methodology  modified  for  use  in  an  Army 
envi ronment . 


Procedure : 

Several  different  validity  tests  were  employed  to  evaluate  the  meth¬ 
odology  and  form  the  basis  for  recommendations  for  improvement.  These  tests 
included  (1)  a  general  examination  of  the  basic  logic  and  appropriateness  of 
the  technique;  (2)  a  reliability  measure  of  bow  well  other  trained  individ¬ 
uals,  given  the  same  data  inputs,  can  derive  the  same  answers  as  an  already- 
completed  test  case;  and  (3)  a  qualitative  accuracy  check  based  on  talking 
to  individuals  who  have  already  applied  the  same,  or  a  similar,  methodology. 


Findings : 

HCM  conforms  well  with  both  the  Army's  MPT  users'  needs  and  other  ac¬ 
cepted  manpower  modeling  techniques.  Audits  of  three  completed  HARDMAN  ap¬ 
plications  reveal  only  a  small  number  of  potential  problem  areas.  The  re¬ 
liability  study  results  conform  well  with  the  problem  areas  uncovered  through 
the  audits.  The  major  problems  discovered  basically  revolve  around  a  need 
to  tighten  up  select  judgmental  sections  of  the  methodology  and  to  improve 
those  aspects  of  the  technique  used  for  making  MPT  cost  estimates.  Recom¬ 
mendations  for  improving  the  methodology  are  provided. 


Utilization  of  Findings: 

The  evaluation  reveals  that  even  with  minor  problems,  the  methodology 
is  sound  and  has  good  near-term  utility  for  the  Army.  Development  of  an 
analyst  guide  is  underway.  Of  equal  importance  is  the  initiation,  by  the 
Soldier  Support  Center,  National  Capital  Region,  of  five  analysis  applications 
to  be  executed  in  FY  85.  These  analyses  will  support  on-going  weapon  devel¬ 
opment  programs  and  contribute  to  the  further  refinement  of  the  methodology. 
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EVALUATION  OF  THE  HARDMAN  (HARDWARE  VS.  MANPOWER) 
COMPARABILITY  METHODOLOGY 


SECTION  I 

INTRODUCTION  AND  SUMMARY 


A.  OBJECTIVES 

The  Jet  Propulsion  Laboratory  (JPL) ,  sponsored  by  the  U.S.  Army  Re¬ 
search  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) ,  was  asked  to 
provide  the  Army  with  an  objective,  independent  assessment  of  the  validity 
and  reliability  of  the  HARDMAN  (Hardware  vs.  Manpower)  Comparability  Meth¬ 
odology  (HCM)  for  projecting  manpower,  personnel,  and  training  (MPT)  require 
ments .  JPL  was  also  asked  to  make  recommendations  for  needed  improvements 
in  the  methodology  that  would  ultimately  help  the  Army  implement  an  accurate 
MPT  system  that  could  be  applied  early  in  the  acquisition  of  new  systems. 

The  following  paragraphs  discuss  the  background  behind  the  Army  man¬ 
power  and  training  issue,  the  major  assumptions  and  constraints  that  estab¬ 
lished  the  framework  of  the  study,  the  evaluation  approach  and  results,  and 
the  overall  structure  of  the  document. 


B .  BACKGROUND 

Over  the  past  several  years,  the  Army  has  recognized  a  critical  man¬ 
power  situation  revolving  around  the  widening  gap  between  increasing  weapon 
complexity  and  declining  manpower  availability.  The  problem  of  declining 
manpower  availability  has  been  aggravated  by  the  apparent  decay  of  entrants’ 
skill  levels.  In  1980,  as  part  of  an  Armywide  interest  in  improving  the 
manpower  problem,  ARI  initiated  an  effort  to  determine  whether  the  Navy's 
HARDMAN  methodology  could  be  applied  to  Army  systems.  The  Army  was  inter¬ 
ested  in  determining  whether  an  existing  manpower,  personnel,  and  training 
projection  technique  could  be  modified  to  provide  early,  accurate  inputs  to 
the  acquisition  process.  The  Army  chose  to  examine  the  HCM  (one  of  several 
methodologies  developed  under  the  auspices  of  the  Navy  HARDMAN  Office)  be¬ 
cause  it  was  an  integrated  MPT  technique  and  had  already  been  used  in  sev¬ 
eral  Navy  acquisition  programs.  The  other  Navy  methodologies  appeared  to  be 
too  narrow  in  scope  or  required  input  data  not  available  under  the  Army's 
existing  data  collection  systems.  Although  these  other  systems  have  not 
been  ruled  out  for  Army  use,  it  seemed  that  in  the  near  term,  the  HCM  would 
provi  !e  the  greatest  payoff.  The  HCM  involves  the  following  six  major  steps 

(1)  Establish  a  consolidated  data  base  composed  of  functional  de¬ 
scriptions  of  the  proposed  and  similar  predecessor  systems  and 
associated  inputs,  such  as  hardware  reliability,  personnel  info 
mation,  costs,  and  training  data.  This  information  is  used  to 


support  succeeding  steps  in  the  analysis.  This  step  is  essential 
because  the  Army  does  not  usually  organize  and  store  data  in 
effective  structure  to  conduct  human  resource  analysis. 

(2)  Determine  the  manpower  requirements  and  skills  necessary  to 
operate  and  maintain  the  system. 

(3)  Determine  training  requirements  likely  to  be  imposed  by  the 
proposed  system. 

(4)  Determine  personnel  requirements  (i.e.,  personnel  flows  that  will 
be  necessary  to  support  the  manpower  requirements). 

(5)  Conduct  an  impact  analysis  to  establish  likely  manpower  and 
training  shortages  and  identify  parts  of  the  system  that  represent 
high-cost  drivers. 

(6)  Perform  a  tradeoff  analysis  to  alter  features  of  the  system  (e.g., 
reliability  or  required  skill  levels)  to  reduce  or  eliminate 
unreasonable  demands. 

Each  of  these  steps  involves  several  judgments  pertaining  to  the 
selection  of  comparable  predecessor  systems,  the  identification  and  assembly 
of  associated  data  on  the  predecessor  systems,  expected  performance  of  the 
proposed  system  as  compared  to  predecessor  systems,  and  required  skills. 
Additionally,  gaps  in  the  supporting  data  often  exist  at  the  conceptual  design 
stage.  At  these  junctions,  expert  judgment  is  exercised  to  merge  the  various 
pieces  of  information  into  a  comprehensive,  consistent  picture  to  be  used  to 
generate  the  personnel  projections.  The  Army's  concern  is  that  these  judgment 
areas  could  act  as  major  error  sources  in  the  manpower  projection. 
Consequently,  JPL  was  asked  to  perform  an  objective  evaluation  of  the 
methodology  to  identify  both  strengths  and  weaknesses  of  the  technique  and 
make  recommendations,  as  appropriate. 


C.  ASSUMPTIONS  AND  CONSTRAINTS 

One  of  the  methodology  validity  tests  involved  a  comparison  of  the  user 
needs  against  the  proposed  outcomes  of  the  projection  scheme.  To  establish 
the  user  requirements,  a  detailed  interview  process  was  designed  to  use  the 
bank  of  experience  already  existing  in  the  Army.  The  major  assumption 
involved  with  the  user  requirements  effort  centered  on  the  user  contacts. 
Because  the  user  contacts  in  the  various  Army  commands  were  established  via 
the  sponsor  and  recommendations  from  the  users  themselves,  it  was  assumed  that 
the  suggested  sources  were  sufficiently  experienced  and  knowledgeable  in  the 
MPT  arena  so  that  their  answers  were  representative  of  the  respective  commands 
in  general.  Even  though  the  sample  size  was  small,  the  consistency  of  the 
responses  suggested  that  the  answers  were  a  good  representation  of  the  Army  as 
a  whole. 
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Several  limitations  were  imposed  on  the  study.  The  size  of  the  sample 
user  population  was  limited  primarily  by  the  available  time  to  complete  the 
evaluation.  Some  of  the  users  contacted  were  more  informed  or  aware  than 
others  about  MPT  processes  or  projection  models  such  as  the  HCM.  Therefore, 
it  was  understood  that  some  of  their  responses  could  reflect  an  already 
established  preference,  depending  on  their  knowledge  of  the  various  elements 
present  in  the  HCM  or  other  MPT  models.  To  circumvent  this  bias,  the 
questionnaire  and  interview  technique  employed  in  this  evaluation  focused  on 
getting  the  user  to  provide  requirements  for  a  "generic"  MPT  projection  system 
for  the  Army. 

It  should  be  noted  that  the  user  requirements  determined  by  this  study 
reflect  the  needs  of  a  present  or  near-future  MPT  system.  Any  changes  to  this 
environment  could  result  in  additional  or  different  requirements  (e.g., 
implementation  of  Integrated  System  Support  (ISS)).  Therefore,  it  is 
important  that  the  Army  consider  periodic  reevaluation  of  its  MPT  requirements 
to  ensure  that  any  in-place  MPT  system  remains  responsive  to  the  needs  of  the 
various  MPT  players. 

Although  it  was  originally  planned  to  do  a  thorough  validity  and 
reliability  evaluation  of  the  methodology,  internal  time  and  budget  limita¬ 
tions  resulted  in  the  sponsor  opting  to  defer  (1)  the  accuracy  check  on  the 
MPT  methodology,  (2)  the  life-cycle  cost  (LCC)  and  methodology  sensitivity 
analysis,  and  (3)  the  quantitative  accuracy  check  on  the  methodology  input 
data  (e.g.,  reliability  and  maintainability  data).  Finally,  other  duty 
commitments  required  that  the  internal  validity  experiment  be  simplified  so 
that  the  Army  participants  could  finish  the  replication  within  a  maximum 
three-week  time  frame. 


D.  SUMMARY  OF  EVALUATION  APPROACH  AND  RESULTS  OF  ANALYSIS 

The  preceding  discussion  outlined  the  various  steps  and  judgments 
necessary  to  exercise  the  HCM.  Because  of  the  breadth  and  complexity  of  MPT 
issues  covered  by  the  technique,  a  multifaceted  evaluation  was  structured  to 
examine  potential  validity  aspects.  The  validity  aspects,  evaluation 
approach,  and  study  results  are  summarized  in  the  following  paragraphs. 


1.  Evaluation  Approach 

As  stated  earlier  in  this  section,  time  and  budget  constraints 
limited  the  original  scope  of  the  study.  Validity  measures  related  to  testing 
model  accuracy  were  deferred  to  the  next  study  phase.  However,  within  the 
imposed  constraints  several  other  validity  tests  were  conducted.  Face 
validity,  a  measure  of  whether  the  model  appears  logical  and  reasonable,  was 
the  first  aspect  investigated.  Although  not  a  rigorous  credibility  test,  a 
face-validity  study  does  afford  some  insights  into  the  general  utility  of  the 
model,  potential  error  sources,  data  availability  problems,  logic  foundation, 
and  possible  ambiguities  in  logic  structure  or  results.  The  approach  for 
performing  the  face-validity  examination  was  fourfold.  First,  a  user 
requirements  study  was  performed  to  establish  the  Army's  needs  within  the  MPT 
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community.  A  total  of  nine  distinct  user  groups  (thirteen  individuals)  were 
interviewed  in  person  to  establish  user  MPT  needs  in  areas ,  ranging  from  the 
basic  design  of  the  MPT  projection  system  to  the  kinds  of  information  desired 
from  the  system  outputs.  A  basic  utility  test  was  then  performed  by  comparing 
each  user  requirement  against  the  same  respective  element  in  the  HCM.  Second, 
existing  DOD  and  Army  directives  were  examined  to  see  whether  there  were  any 
established,  accepted  MPT  techniques  presently  in  use  that  could  be  used  to 
confirm  the  general  logic  structure  and  data  foundation.  Third,  a  larger 
literature  search  was  conducted,  encompassing  other  government  agencies  and 
the  private  sector.  Again,  the  object  was  to  develop  a  firm  basis  for 
evaluating  the  logic  and  data  support  structures  of  the  HCM,  as  well  as 
examining  potential  sources  for  error.  Last,  detailed  audits  were  conducted 
on  three  applications  of  the  methodology.  Independent  analysts  attempted  to 
replicate  major  empirical  and  judgmental  steps  in  each  application  with  the 
intent  of  examining  overall  logic,  potential  sources  of  errors,  data 
manipulation  or  availability  problems,  and  ambiguities  in  how  the  projections 
were  derived  and  interpreted. 

Another  test,  internal  reliability,  was  employed  to  get  an  understanding 
of  the  ability  of  others  to  successfully  apply  the  technique  and  obtain 
consistent  answers.  An  internal  validity  test  is  essentially  a  measure  of  the 
variance  in  model  outputs  when  a  group  of  independent  analysts  replicate  an 
already  completed  application  given  the  same  inputs.  Ultimately,  the 
objective  is  to  establish  the  clarity  and  succinctness  of  the  model,  as  well 
as  the  ability  to  have  any  reasonably  trained  individual  perform  the  process 
reliably.  An  actual  Army  system  was  selected  for  the  internal  validity  test. 

A  group  of  seven  individuals  (four  civilians  and  three  Army  officers)  were 
initially  trained  in  the  execution  of  the  HCM  and  then  provided  with  data 
packages  that  allowed  them  to  conduct  a  "limited"  manpower  and  personnel 
assessment  on  the  proposed  system  (i.e.,  because  of  other  duty  commitments  and 
resultant  time  restrictions,  only  four  major  components  of  the  system  were 
studied) . 

It  was  decided  that  an  attempt  should  be  made  to  examine  model  accuracy 
from  at  least  a  qualitative  standpoint.  To  this  end,  several  individuals  in 
both  the  Air  Force  and  Navy,  who  had  experience  with  applications  of  the  same 
(or  similar)  MPT  model,  were  contacted  for  information  pertaining  to  any  tests 
or  actual  comparative  field  data  that  might  indicate  the  relative  accuracy  of 
the  methodology. 

Finally,  the  last  validity  aspect  examined  was  an  event-series  test. 
Event-series  validity  is  basically  a  measure  of  how  accurately  a  simulation 
must  reproduce  a  real-world  event.  A  top-level,  life-cycle,  cost-sensitivity 
analysis,  keying  on  the  manpower  variable,  was  employed  as  the  tool  to  achieve 
this  measure.  The  required  accuracy  of  the  manpower  variable  was  determined 
by  performing  iterations  of  the  life-cycle  cost  projection  (using  an  example 
acquisition)  until  all  other  dependent  variables,  manning,  and  total 
life-cycle  cost  fell  within  prescribed  error  bounds.  This  measure  was 
particularly  important  for  obtaining  a  further  understanding  of  the  model 
usefulness  as  a  decision  making  tool. 
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Throughout  all  of  these  validity  exercises  several  strengths  and  parallel 
problem  areas  surfaced.  The  complete  results  of  the  evaluation  are  summarized 
in  the  following  discussion. 

2.  Summary  of  Results 

Table  1-1  provides  an  overall  summary  of  the  study  results.  Each 
of  the  above  evaluation  areas  is  discussed  briefly  in  the  following  paragraphs. 


Table  1-1.  Net  Results  of  HARDMAN  Evaluation 


Evaluation  Areas  Net  Results 

User  Requirements  Technique  complied  with  majority  of  mandatory 

user  requirements 

Operational  Analysis 

Comparison  with  Methodology  conformed  with  other  known,  accepted 

other  MPT  methods  MPT  modeling  schemes,  and  data  foundations 

Audits  Methodology  demonstrated  sound  logic  and 

reasonable  results  for  &5X  of  test  issues 
examined  (remaining  issues  considered  reparable 
in  near  term) 

Test  group  correctly  replicated  half  of  selected 
test  points  (two  of  the  four  remaining  test 
points  found  to  be  repeatable  after  clarification) 

Qualitative  Two  of  nine  individuals  having  experience  with 

Accuracy  Check  HARDMAN  applications  indicated  a  rough  accuracy 

of  80-90Z  with  actual  manning  requirements; 
remaining  individuals  had  insufficient  experience 
to  comment 

Manpower  variable  found  to  be  least  sensitive  of 
all  life-cycle  cost  variables,  demonstrating  good 
utility  of  existing  methodology  even  without 
suggested  near-term  improvements 


Reliability  Analysis 
Internal  reliability 


Event  Series  Validity 


t 
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a.  User  Requirements  Analysis.  The  user  requirements  analysis 
generated  a  total  of  twenty-two  mandatory  MPT  system  requirements.  Thirteen 
of  the  mandatory  requirements  applied  to  system  outputs,  and  the  remaining 
nine  requirements  were  related  to  system  design.  An  additional  four  .require¬ 
ments  were  given  lower  priority  by  the  users  and  were  therefore  listed  as 
"desirable."  The  major  thrust  of  the  user  needs  revolved  around  the  key  design 
drivers  of  affordability,  supportability,  and  sustainability.  By  comparison, 
it  was  found  that  the  HCM  complied  with  seventeen  of  the  twenty-two  mandatory 
requirements.  The  areas  of  noncompliance  were  as  follows: 

(1)  Not  indicating  the  accuracy  of  the  MPT  projection  system  to  aid 
budget  control  and  program-level  decisions. 

(2)  Not  including  demographic  data  such  as  age,  sex,  size,  etc.,  as 
part  of  the  man-machine  matching  and  support  process. 

(3)  Not  considering  the  demands  of  other  acquisitions  on  the  available 
manpower  pool  (i.e.,  the  HCM  still  focuses  primarily  on  the 
individual  equipment  system). 


(4)  Not  performing  a  complete  life-cycle  cost  tradeoff  analysis  (i.e., 
the  HCM  focuses  only  on  MPT  costs). 


(5)  Not  indicating  weaknesses  in  the  MPT  analysis  along  with 
respective  resolution  actions. 

Overall,  it  seems  that  the  HCM  is  reasonably  responsive  to  the  Army's 

needs . 


b.  Operational  Analysis.  The  literature  search  and  audit 
aspects  of  the  face  validity  study  were  jointly  conducted  under  the 
operational  analysis  portion  of  the  evaluation.  This  portion  of  the  overall 
study  was  broadly  termed  "operational  analysis"  because  the  ensuing 
investigation  required  a  rigorous  exercising  of  the  HCM.  The  review  and 
comparison  of  related  DOD  and  Army  directives  provided  three  major  findings: 
(1)  DOD  system  acquisition  guidelines  stated  a  requirement  to  consider  MPT  in 
the  design  process  but  did  not  appear  to  afford  a  means  of  projecting  MPT 
requirements;  (2)  the  Army  Manpower  Authorization,  Criteria  (MACRIT)  guide  did 
provide  a  simple  manpower  projection  algorithm;  and  (3)  the  HCM  appeared  to  be 
designed  around  the  MACRIT  manpower  algorithm,  with  its  outputs  generally 
organized  to  conform  with  broader  DOD  acquisition  directives.  These  insights, 
although  serving  as  good  initial  data  points,  only  addressed  a  fraction  of  the 
total  methodology.  The  follow-up  government  and  private-sector  literature 
searches  were  far  more  intensive.  Nineteen  other  MPT  models  and  papers, 
encompassing  all  aspects  of  the  HCM,  were  studied.  Generally,  it  was 
discovered  that  the  other  modeling  approaches  were  reasonably  equivalent  in 
their  assumptions,  algorithms,  and  data  inputs.  In  one  area,  the  personnel 
pipeline  projection,  the  HCM  approach  was  found  to  be  stronger  because  it 


employed  a  modeling  technique  that  reduced  the  need  for  dif ficult-to-obtain 
data  such  as  present  and  future  manpower  availabilities.  On  the  negative  side, 
it  seemed  that  there  were  four  distinct  areas  where  potential  inaccuracies 
could  be  interjected:  (1)  The  use  of  the  MACRIT  manpower  algorithm  could 
result  in  early  or  late  life-cycle  manpower  projection  errors  unless  system 
deployment  and  phase-out  rates  are  considered;  (2)  non-consideration  of  major 
socio-economic  factors  such  as  the  draft,  economy,  change  in  federal  govern¬ 
ment  administrations,  or  massive  military  reorganizations  could  influence  the 
attrition  and  retention  characteristics  of  personnel  career  paths;  (3)  the 
non-use  of  some  type  of  long-term  manpower  availability  projection  to  indicate 
shortages  at  the  time  of  system  deployment  could  result  in  sizeable  personnel 
errors;  and  (4)  the  lack  of  a  complete  life-cycle  cost  model  could  prevent  the 
full  range  of  potential  cost  and  system  design  tradeoffs  from  being  provided 
to  the  decision  makers. 

The  last  aspect  of  the  face-validity  analysis,  the  audits,  revealed  that 
the  model  successfully  passed  eighty-five  percent  of  the  test  issues  examined. 
The  remaining  potential  problem  areas  discovered  were  (1)  the  lack  of  a 
structured  process  of  selecting,  adjusting,  and  manipulating  the  source  data 
could  sizeably  affect  the  accuracy  and  credibility  of  the  projections;  (2)  the 
use  of  a  single  reference  system  (composed  of  similar  predecessor  components) 
to  model  several  widely  varying  proposed  designs  could  also  affect  the 
accuracy  and  credibility  of  the  results;  (3)  the  absence  of  an  adjustment  to 
the  complete  predecessor  historical  data  to  consider  the  design  improvements 
of  the  proposed  system  means  that  it  would  not  be  feasible  to  make  a  valid 
comparison  between  real  world  performance  and  the  often  overly-optimistic 
projection  of  the  contractor;  (4)  the  lack  of  a  structured  process  for 
approximating  the  value  of  peculiar  design  changes  or  induced  human  error  as 
related  to  component  failure  rates  presents  a  means  of  interjecting  computa¬ 
tional  mistakes  as  well  as  reducing  both  the  audit  and  usability  capacities  of 
the  method;  (5)  the  absence  of  indicators  for  data  quality  and  projection 
accuracy  could  affect  the  credibility  and  usefulness  of  the  results;  and  (6) 
the  complexity  of  the  technique  requires  the  use  of  a  well-seasoned,  multi¬ 
disciplinary  team  which,  in  turn,  could  impact  the  ability  of  others  to  apply 
the  method. 


c.  Reliability  Analysis.  The  last  parts  of  evaluation,  the 
internal  and  event-series  validity  tests  and  qualitative  accuracy  check,  were 
performed  under  the  reliabiity  analysis.  This  was  done  because  all  of  these 
validity  aspects  related  to  both  the  internal  and  external  reliability  of  the 
process.  The  results  of  the  internal  validity  check  were  closely  aligned  with 
the  audit  findings.  As  a  whole,  the  test  group  discovered  that,  given  the 
same  inputs,  it  could  not  replicate  the  original  SINCGARS  results.  Although 
all  of  the  study  group  seemed  to  understand  roughly  half  of  the  early  steps  in 
the  process  reasonably  well,  they  had  problems  with  (1)  developing  a  reference 
system  that  jointly  met  the  functional  requirements  and  was  representative  of 
the  two  widely  varying  contractor  designs;  (2)  not  understanding  the  thought 
process  for  determining  impacts  of  design  differences,  and  how  and  when  to 
perturb  historical  predecessor  data  to  model  components  of  the  proposed 
system;  and  (3)  not  comprehending  the  exact  process  of  matching  or  selecting 
skills.  After  items  (1)  and  (3)  were  clarified,  the  study  group  had  a  fairly 
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good  grasp  of  the  majority  of  the  early  steps  in  the  process.  The  test 
subjects  uniformly  agreed  that  none  of  them  singularly  had  enough  training  or 
educational  background  to  exercise  the  complete  methodology  with  any 
confidence. 

The  major  finding  in  the  event-series  validity  study  was  that  the  MPT 
projections  could  easily  vary  by  plus  or  minus  fifteen  percent  without  causing 
major  problems  at  the  upper  DOD  budget  and  cost  control  management  levels. 
Component  usage  rate  was  found  to  have  the  greatest  influence  on  causing 
unacceptable  errors  in  manpower  projections.  Although  it  was  jointly  dis¬ 
covered  that  there  is  still  not  sufficient  historical  experience  to  make  a 
firm  qualitative  accuracy  judgment  on  the  process,  the  low  sensitivity  finding 
on  the  manpower  variable  was  a  promising  indicator  of  the  utility  of  the  model 
even  with  its  minor  drawbacks. 


d.  Conclusions  and  Recommendations.  In  conclusion,  it  was  felt 
that,  overall,  the  HCM  appeared  to  be  reasonably  sound.  The  flaws  discovered 
did  not  appear  to  be  irreparable.  The  major  near-term  recommendations 
suggested  to  tighten  up  the  problem  areas  were  as  follows:  (1)  Establish 
subjective  criteria  for  rating  the  quality  of  various  data  sources;  (2) 
provide  an  indication  of  the  credibility  of  the  projections;  (3)  establish  a 
top-level  method  for  comparing  the  technology  of  the  reference  and  proposed 
systems  on  an  equal  basis;  (4)  establish  solid  guidelines  and  information 
sources  for  building  reference  systems  and  consider  the  use  of  two  reference 
systems  to  model  two  widely  varying  contractor-proposed  designs;  (5)  firm  up 
guidelines  for  the  functional  requirements  level  of  indenture;  (6)  provide  a 
better  structure  for  MOS  and  skill  selection;  (7)  periodically  update  manpower 
projections  (every  two  to  four  years)  over  a  system's  life  cycle  to  offset 
possible  end-point  errors;  (8)  provide  a  more  structured  tradeoff  process;  and 
(9)  standardize  the  assessment  of  study  weaknesses  and  resolution  actions. 

The  intermediate  term  (two  years  from  the  present)  priorities  were 

(1)  project  manpower  availability  out  to  the  actual  system  deployment  date; 

(2)  incorporate  a  more  complete  life  cycle  cost  tradeoff  analysis;  (3)  firm  up 
ranges  for  perturbation  values,  induced  failures,  and  indirect  productivity 
factors . 

Far-term  recommendations  (for  the  three-  to  five-year  time  frame 
following  the  present)  included  (1)  consideration  of  other  emerging  system 
manpower  demands;  (2)  development  of  a  quantitative  means  to  differentiate 
between  critical  and  non-critical  tasks;  (3)  expansion  of  the  demographic  data 
base;  and  (4)  consideration  of  major  socioeconomic  fluctuations.  Potential 
solutions  for  all  of  the  preceding  recommendations  were  also  provided. 

In  addition  to  these  recommendations,  it  was  also  suggested  that,  based 
on  the  audit  and  reliability  findings,  future  HARDMAN  training  programs  solicit 
a  multidisciplinary  team  of  individuals  with  backgrounds  in  the  military, 
engineering,  information  systems  and  data  management,  and  personnel.  Further, 
combinations  of  two  of  more  of  these  disciplines  in  each  individual  as  part  of 
the  overall  multidisciplinary  team  seem  to  form  the  best  foundation  for  accu¬ 
rately  exercising  the  methodology. 


E. 


STRUCTURE  OF  DOCUMENT 


This  document  is  structured  so  that  a  brief  history  of  both  the  HARDMAN 
effort  and  Army  involvement  in  HARDMAN  is  provided  first  in  Section  II  as  a 
basis  for  the  study.  Section  III  discusses  the  overall  design  of  the  analysis 
and  broadly  explains  the  purpose  and  content  of  the  various  validity  studies. 
Sections  IV  through  VI  provide  the  detailed  results  of  the  methodology  evalua¬ 
tion  in  the  major  areas  of  user  requirements,  face  validity  (operational 
analysis),  and  reliability  (internal  and  event-series  validity).  Finally, 
Section  VII  discusses  the  implications  of  the  results  and  provides  the  overall 
assessment  of  the  technique  with  recommendations. 
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SECTION  II 


HARDMAN  BACKGROUND 


A.  OVERVIEW 

The  previous  section  provided  a  brief  summary  of  the  evaluation  approach 
and  results.  As  stated  earlier  in  Section  I,  the  Army  has  been  concerned 
about  the  widening  gap  between  growing  technology  and  decreasing  availability 
of  needed  skills.  Manpower  guidelines  already  exist  through  DOD  documents 
such  as  the  5000  series  and  Military  Standard  (MIL-STD)  1388.  These  documents, 
which  establish  acquisition  and  support  procedures  for  new  systems,  address 
the  requirement  to  have  a  proper  man-machine  balance  but  do  not  specify 
techniques  to  accomplish  the  balance.  Given  the  ever-widening  technology  and 
skills  gap,  the  Army  has  been  pressed  to  develop  clear,  accurate  manpower 
projection  techniques  to  fill  the  void  in  existing  DOD  acquisition  guidelines. 
The  following  paragraphs  provide  a  brief  summary  of  the  history  and  progression 
of  events  leading  up  to  the  Army's  decision  to  develop  an  appropriate  MPT  pro¬ 
jection  system. 


B.  HARDMAN  HISTORY 


The  weapon  system  acquisition  process  has  evolved  over  a  long  period  of 
time.  The  overall  system  acquisition  strategy  suggests  that  the  number  of 
competing  alternatives  should  be  narrowed  by  eliminating  those  concepts  no 
longer  considered  viable  as  a  function  of  four  major  measures  (1,2):  (1)  the 

ability  of  a  design  to  meet  the  proposed  mission;  (2)  the  compatibility  of  the 
design  with  the  anticipated  threat;  (3)  a  comparison  of  each  concept  with 
operational  requirements;  and  (4)  a  comparison  of  each  design's  threshold 
values  (i.e.,  cost,  performance,  readiness,  and  supportability  impacts). 

These  measures  are  employed  throughout  the  system  development  process.  The 
development  process  consists  of  four  phases  containing  three  major  milestones 
or  acquisition  decision  points  (2): 

(1)  Phase  I  (Milestone  I)  -  Concept  development. 

(2)  Phase  II  (Milestone  II)  -  Concept  demonstration  and 

validation. 


(3)  Phase  III  (Milestone 

(4)  Phase  IV 


III)-  System  development. 

-  System  production 
and  deployment. 


At  the  end  of  each  phase  or  milestone,  a  go/no-go  decision  is  made  by 
the  Army  System  Acquisition  Review  Council  (ASARC)  and  Defense  System  Acquisi¬ 
tion  Review  Council  (DSARC)  as  to  whether  to  proceed  to  the  next  milestone 
(1,2).  A  key  consideration  in  all  of  the  preceding  measures  and  subsequent 
decision  points  is  the  integration  of  manpower,  personnel,  and  training  (MPT) 
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into  new  designs.  Recent  studies  (3,  4)  by  some  of  the  services  (i.e.,  Air 
Force  and  Navy)  indicate  that  operations  and  support  costs  can  consume  from  40 
to  502  of  the  total  life-cycle  cost  of  a  weapon  system.  Additionally,  the 
manpower  component  of  the  support  portion  of  life-cycle  cost  can  consume  as 
much  as  602.  These  studies  underline  the  growing  importance  of  MPT. 

Although  the  Air  Force  was  the  first  service  to  develop  a  major  thrust 
to  resolve  the  MPT  dilemma,  the  actual  formalization  of  the  HARDMAN  concept, 
as  presently  defined,  started  with  the  creation  of  Navy's  HARDMAN  Development 
Office  in  1977  (5).  The  Navy's  HARDMAN  effort  was  initiated  partially  because 
of  studies  similar  to  References  3  and  4  and  to  the  fact  that  present  DOD 
directives  do  not  adequately  define  methods  for  projecting  MPT  requirements 
for  new  acquisitions.  The  first  Navy  HARDMAN  study,  initiated  in  1975,  was 
funded  by  the  Chief  of  Naval  Operations  (CNO)  Studies  and  Analysis  Program. 
This  study  established  that  (1)  systems  were  being  designed  without  adequate 
attention  given  to  MPT  costs  and  (2)  logistics  support  planning  (and  in 
particular  MPT)  for  new  acquisitions  was  inadequate.  The  CNO  study  outlined 
the  following  solutions: 

(1)  Develop  analytical  tools  and  methods  for  MPT  to  influence  the 
weapon  system  acquisition  process. 

(2)  Develop  an  MPT  information  system. 

(3)  Revise  existing  directives  governing  the  acquisition  process  to  be 
more  sensitive  to  MPT. 

As  a  result  of  the  CNO  study  and  some  aggressive  management,  the  Navy 
established  the  HARDMAN  office  and  set  about  developing  a  modeling  system 
composed  of  four  linked  components  consistent  with  the  aforementioned  CNO 
guidelines  and  tailored  to  the  various  design  phases. 

The  linked  modeling  system,  which  is  the  present  HARDMAN  design, 
basically  consisted  of  the  following: 

(1)  A  manpower  projection  model. 

(2)  A  training  resource  assessment. 

(3)  A  life-cycle  cost  analysis. 

(4)  A  HARDMAN  information  system. 

The  manpower  projection  link  of  the  system  (HCM)  was  designed  by 
Dynamics  Research  Corporation  (DRC)  and  was  developed  around  the  concept  of 
using  information  from  eimilar  predecessor  technology  as  a  basis  for  manpower 
projections  (6).  Drawing  on  an  extensive  historical  data  base  as  well  as  the 
new  design  inputs,  the  manpower  for  the  proposed  system  is  estimated  by 
altering  the  historical  data  as  a  function  of  the  peculiar  design  improvements 
and  fitting  these  results  into  a  simple  manpower  equation  consisting 
principally  of  total  workload  manhours  and  total  systems  to  be  procured.  The 
HCM  proceeds  to  establish  required  skills,  training,  MP  costs,  and  potential 
personnel  shortages  through  the  remaining  steps  in  the  methodology.  MPT 
estimates  are  then  refined  as  the  input  data  matures  through  subsequent  design 
stages . 
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The  PACER  Corporation  developed  the  training  resource  link  (HTRRDM), 
designed  to  provide  a  checklist  and  straightforward  algorithms  for  defining 
elements  such  as  training  sites,  instructors,  instructional  devices,  course 
material,  and  training  costs  (7). 

The  life-cycle  cost  link  of  the  HARDMAN  system  (HLCCM)  was  developed  by 
the  Assessment  Group  firm  and  designed  in  two  levels  of  detail  to  be  com¬ 
mensurate  with  the  different  stages  of  design  development.  Drawing  on  standard 
life-cycle  cost  algorithms,  a  skill-cost  model,  and  designer  inputs,  the  tech¬ 
nique  provides  both  the  total-system,  life-cycle  cost  and  the  ability  to 
quickly  perform  life-cycle  cost  tradeoffs  between  various  logistic  elements  (8). 

The  last  system  link,  the  HARDMAN  Information  System  (HIS),  was  designed 
by  Ackman  Associates.  Although  not  completed,  the  HIS  is  structured  to  be  the 
repository  for  all  predecessor  system  MPT  and  life-cycle  cost  data  as  well  as 
data  for  new  potential  acquisitions. 

The  Navy  effort  has  considerable  value  for  several  reasons.  Most 
important  is  the  clear  demonstration  of  the  necessity  to  bond  MPT  securely 
with  the  acquisition  community.  Another  clear  value  of  the  Navy's  effort 
revolves  around  the  many  manpower  models  developed  by  DOD  and  the  various 
services  over  the  past  several  years.  These  models  generally  have  only 
addressed  select  aspects  of  the  MPT  problem  such  as  manpower  flows  in  and  out 
of  various  skill  areas  or  total  manpower  requirements.  The  Navy's  thrust  has 
been  to  develop  a  total  MPT  system,  which  extends  beyond  these  smaller 
studies.  Additionally,  the  Navy's  program  has  produced  a  large  bank  of 
experience  and  MPT  tools. 


C.  ARMY  INVOLVEMENT  IN  HARDMAN 

Like  the  Navy,  the  Army  has  been  observing  similar  problems  with  the 
mismatch  between  advancing  technology  and  decreasing  availability  of  skills. 

In  1980,  the  Army  Research  Institute  (ARI)  initiated  an  effort,  in  support  of 
the  Army-wide  interest  in  the  system  manning  problem,  to  establish  whether  the 
Navy  HARDMAN  effort  could  be  tailored  for  use  on  new  Army  acquisitions . 
Recognizing  that  approximately  seventy  five  percent  of  the  system  acquisition 
cost  is  committed  by  Phase  II  of  the  development  cycle,  the  Army  was  also 
interested  in  determining  whether  the  technique  could  provide  early, 
reasonably  accurate  MPT  estimates.  As  indicated  in  the  preceding  section, 
there  was  considerable  overlap  between  the  comparability,  training,  and 
costing  links.  Because  it  appeared  that  the  HCM  was  the  most  encompassing  and 
mature  of  the  MPT  models,  ARI  decided  to  perform  some  test  applications  of 
this  technique  as  well  as  a  validation  of  the  methodology.  To  date,  the  Army 
has  funded  four  test  applications:  Corps  Support  Weapon  System  (CSWS), 
Division  Support  Weapon  System  (DSWS),  Remotely  Piloted  Vehicle  (RPV),  and 
Single  Channel  Ground  Airborne  Radio  System  (SINCGARS).  The  study  performed 
in  this  document  represents  the  first  segment  of  the  Army's  validation  effort 
and  draws  on  the  results  of  three  of  the  test  applications.  The  HCM  presently 
represents  a  possible  near-term  solution  to  the  Array’s  MPT  dilemma. 


D.  DESCRIPTION  AND  INTENDED  USE  OF  THE  ARMY  HARDMAN  SYSTEM 

The  Army  HARDMAN  system  is  presently  emphasizing  only  the  HCM.  The 
technique  consists  of  six  major  steps  as  shown  in  Figure  2-1  (6).  Step  1, 
Establish  Consolidated  Data  Base,  includes  the  development  of  functional 
descriptions  of  the  alternate  systems  and  the  collection  of  the  input  data 
(hardware,  personnel,  training,  and  cost)  needed  to  support  the  succeeding 
analyses.  Step  2,  Determine  Manpower  Requirements,  refers  to  the  process  of 
identifying  the  number  and  skills  of  soldiers  necessary  to  operate  and 
maintain  the  system.  Step  3,  Determine  Training  Requirements,  is  done  in 
parallel  with  Step  4  and  requires  the  specification  of  the  numbers  and  types 
of  instructors,  courses,  media,  and  training  devices,  etc.,  which  are 
necessary  to  accommodate  the  new  equipment.  Step  4,  Determine  Personnel 
Requirements,  addresses  the  activity  of  identifying  the  peculiar  occupational 
speciality  requirements.  Step  5,  Conduct  Impact  Analysis,  focuses  on  the 
activity  of  comparing  assessed  demands  (from  Steps  2-4)  and  available  supplies 
and  notes  unreasonable  resource  demands  in  terms  of  both  manpower  and  training 
costs  and  personnel.  Step  6,  Tradeoff  Analysis,  refers  to  the  activity  of 
altering  features  of  the  system  and  repeating  Steps  1  through  5  to  reduce  or 
eliminate  unreasonable  demands. 

Within  each  step  there  are  several  judgments  required  that  are  basically 
satisfied  by  using  experience  from  the  new  system  designers  or  a  group  of 
outside  experts  (6).  For  example,  in  Step  1,  the  evaluator(s)  is  required  to 
interpret  documents  and  use  personal  judgment  in  establishing  the  functional 
requirements  for  the  new  system  and  in  selecting  similar  predecessor 
components  that  most  closely  meet  these  requirements.  In  Step  2,  generic 
kinds  of  tasks  must  be  defined  and  assigned  to  the  various  echelons  of 
maintenance  (i.e.,  organizational  level  in  the  field  and  intermediate  level 
removed  from  the  field).  Step  2  also  requires  the  evaluator(s)  to  make 
educated  guesses  as  to  the  split  between  inherent  versus  induced  component 
failures,  design  differences,  selection  and  assignment  of  skills  for  the 
various  tasks,  and  probable  system  and  component  usage  rates.  In  Step  3,  the 
analyzer  must  decide  which  kinds  of  predecessor  training  resources  are 
applicable  to  the  proposed  system  and,  where  no  training  scenario  exists, 
generate  new  training  resource  requirements.  Steps  4  and  5  involve  judgments 
about  the  movement  and  availability  of  people  both  within  various  skills  and 
the  Army  as  a  whole  and  also  require  the  development  of  options  to  offset  high 
demands.  Step  6  does  not  require  any  major  judgments  other  than  a  selection 
of  different  operating,  reliability,  or  manning  scenarios  that  represent  a 
modified  system  designed  to  offset  high  manpower  demands  or  costs. 


In  reviewing  the  Army  MPT  problem  outlined  earlier  in  this  section,  it 
appears  that  a  technique  such  as  the  HCM  could  certainly  contribute  to 
resolving  most  of  the  man-machine  mismatches  with  new  acquisitions. 
Ultimately,  it  is  the  Army's  primary  intent  to  use  the  results  of  such  an 
approach  to  influence  the  ASARC,  DSARC,  and  system  program  management 
communities  to  be  more  supportive  of  the  MPT  portion  of  system-integrated 
logistics  support.  At  a  secondary  level,  it  is  also  the  Army's  desire  to 
examine  techniques  such  as  the  HCM  with  the  intent  of  determining  the 
feasibility  of  enhancing  and  managing  its  own  MPT  system.  For  both  the  ASARC 
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decision  and  internal  MPT  management  objectives  it  is  important  that  the  Army 
establish  a  credible  MPT  projection  scheme.  It  then  follows  that  any 
anticipated  MPT  projection  techniques  be  validated  before  being  approved  for 
Army-wide  use.  The  following  section  describes  the  structure  for  the  HCM 
validity  evaluation  and  sets  the  foundation  for  the  final  results  provided 
later  in  the  report. 
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SECTION  III 

STRUCTURE  OF  HARDMAN  ANALYSIS 


A.  OVERVIEW 

The  previous  section  described  how  important  it  is  for  the  Army  to 
develop  a  credible,  comprehensive  MPT  system.  There  are  several  different 
kinds  of  validity  tests  to  consider  when  establishing  the  true  credibility  of 
a  modeling  technique.  For  example,  some  kinds  of  mathematical  models  may  obey 
other  already  proven  algorithmic  or  empirical  relationships.  Validating 
techniques  using  this  method  is  rather  simple.  However,  the  problem  becomes 
more  difficult  when  a  proposed  simulation  is  projecting  the  future  based  on 
experiential  judgment  and  scant,  or  non-existent,  historical  evidence.  As 
shown  in  Section  II,  the  proposed  Army  MPT  methodology  not  only  fits  this 
description  but  is  further  complicated  by  the  need  to  draw  on  a  considerable 
amount  of  data  and  data  sources.  In  situations  such  as  this  there  are  several 
different  kinds  of  validity  tests  that  can  be  employed  to  test  "first-time" 
models.  In  this  section  the  various  kinds  of  validity  tests  are  first 
described,  followed  by  a  detailed  discussion  of  the  structure  of  each  portion 
of  the  evaluation  in  the  context  of  the  validity  tests. 


B.  VALIDITY  TESTING 

The  two  important  outcomes  of  testing  the  validity  of  a  simulation  or 
projection  are  the  degree  of  credibility  of  the  model  and  results,  and  the 
respective  usefulness  of  the  results.  The  credibility  of  a  projection 
methodology  revolves  around  several  operations  research-type  measures.  Two 
very  important  validity  measures  are  as  follows: 

(1)  Variable  parameter  validity  -  a  measure  of  how  well  the  simulation 
parameters  compare  with  their  respective  counterparts  in  the  real, 
or  observable,  world;  and  whether  the  parameters,  when  varied  over 
an  expected  range,  effect  the  results  in  the  direction  of  known 
historical  data. 

(2)  Hypothesis  validity  -  a  measure  of  how  well  various  relationships 
within  the  model  and  its  outcomes  correspond  with  the  real  world. 

Both  of  these  validity  measures  fit  under  generic  tests  of  "model 
accuracy."  As  stated  earlier  in  Section  I“B,  the  detailed  accuracy 
examination  has  been  postponed  to  a  later  phase. 

Within  the  constraints  of  this  study,  the  following  validity  tests  were 
exercised : 

(1)  Face  validity  -  a  general  indication  of  whether  the  model  is 

logical,  credible,  and  the  results  reasonably  based  on  the  inputs. 
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This  indication  can  be  obtained  by  talking  to  individuals 
experienced  with  the  same,  or  similar,  models;  comparing  the 
structure  of  the  model  to  other  established  simulations;  or 
tracing  and  evaluating  test  case  results  for  reasonableness. 

(2)  Internal  reliability  -  a  measure  of  the  variance  in  model  outputs 
when  replicated  by  a  group  of  independent  assessors  given  the  same 
inputs . 

(3)  Event  series  validity  -  It  is  understood  that  inherent  within  the 
design  of  any  model  is  the  inability  to  duplicate  exactly  the  real 
world.  Event  series  validity  is  a  measure  of  how  accurately  the 
simulation  must  reproduce  real-world  events.  This  test  is 
extremely  important  because  it  relates  to  both  the  credibility  and 
utility  of  the  results  as  viewed  by  the  people  using  the  outputs 
for  their  decision  process  (i.e.,  the  ASARC  and  Program  Managers). 

In  the  following  paragraphs,  the  various  analytic  procedures  employed  in 
evaluating  the  Army  MPT  methodology  are  discussed  in  terms  of  the  preceding 
validity  tests. 

C.  USER  REQUIREMENTS  ANALYSIS 

One  of  the  more  obvious  tests  of  face  validity  is  a  straightforward, 
initial  comparison  of  the  user  needs  against  the  outputs  of  the  model.  By 
querying  knowledgeable  individuals  in  the  Army  MPT  arena  for  problems,  and 
their  respective  needs  to  resolve  those  problems,  it  was  possible  to  evaluate 
whether  the  proposed  methodology  could  illuminate  the  key  personnel  issues. 

The  effort  designed  to  establish  the  relevance  of  the  HCM  MPT  system  to  the 
Army's  needs  was  fourfold: 

(1)  Establish  a  list  of  experienced,  interested  individuals  from  the 
Army  MPT  conanunity  and  attempt  to  draw  on  individuals  from  as  many 
participating  commands  as  possible  to  establish  a  broad  user  base. 

(2)  Design  a  comprehensive  set  of  questions  to  probe  both  major  MPT 
problem  areas  and  needed  solutions  and/or  tools  to  resolve 
personnel  issues. 

(3)  Obta  in  answers  to  these  questions  via  personal  interviews  with 
each  individual. 

(4)  Compare  the  results  against  the  applicable  portions  of  the 
methodology  to  establish  whether  HARDMAN  is  reasonably  responsive 
to  the  stated  issues. 

Each  step  in  the  above  user  analysis,  along  with  both  the  user  require¬ 
ments  and  relevance  results,  is  discussed  in  greater  detail  in  Section  IV. 


D. 


OPERATIONAL  ANALYSIS 


As  a  continuation  of  the  face-validity  evaluation,  it  is  also  useful  to 
examine  applications  of  the  methodology  as  a  means  of  establishing  credibility. 
In  this  study  the  examination  encompasses  both  actual  test  cases  and  comparison 
with  other  accepted  or  proven  MPT  projection  techniques.  The  structure  and 
results  of  both  of  the  aforementioned  examinations  are  provided  in  greater 
detail  later  in  Section  V.  However,  as  a  basis  for  the  outcomes  provided  in 
Section  V,  it  is  important  to  note  that  the  results  afforded  insight  into 
several  credibility  issues.  First,  the  results  provided  information  on  how 
rational  the  model  was.  For  example,  the  test-case  audits  were  established  to 
examine  the  consistency  and  reasonableness  of  the  results  relative  to  the 
inputs.  Similarly,  by  comparing  the  technique  against  other  accepted  models, 
it  was  possible  to  see  whether  the  basic  assumptions  and  logic  structure 
appeared  sound.  Second,  the  audit  results  provided  useful  information  on  the 
complexity,  and  subsequent  utility,  of  the  model.  Third,  the  audits  provided 
an  understanding  of  whether  the  model  logic  and  interpretation  of  the 
projections  were  perhaps  ambiguous  or  masked,  due  to  model  over-complexity. 
Fourth,  the  audits  indicated  the  presence  and  relative  occurrence  of  error 
sources  in  the  methodology.  Last,  the  use  of  both  test  cases  and  comparisons 
with  other  MPT  models  highlighted  potential  problems  in  input  data  quality  and 
availability. 

E.  RELIABILITY  ANALYSIS 

An  examination  of  the  "reliability"  or  accuracy  of  the  methodology  and 
its  projections  was  the  original  intent  of  this  portion  of  the  evaluation. 
Although  it  was  possible  to  retain  part  of  the  original  reliability  analysis 
scope,  time  and  funding  constraints  required  the  actual  accuracy  effort  to  be 
postponed  until  the  next  evaluation  phase.  The  portions  retained  for  this 
part  of  the  evaluation  address  the  issues  of  internal  reliability,  a  pre¬ 
cursory  accuracy  examination  using  the  experience  of  individuals  who  have 
already  applied  the  HCM,  or  a  similar  methodology,  and  event-series  validity. 

For  the  internal  reliability  test,  it  was  decided  to  use  one  of  the  Army 
test  cases  (i.e.,  SINCGARS)  and  design  a  controlled  experiment  to  establish 
whether  a  group  of  individuals,  initially  trained  to  use  the  methodology, 
could  both  replicate  the  original  answers  and  demonstrate  a  low  variance  in 
answers  among  themselves.  A  group  of  seven  individuals  (four  civilians  and 
three  career  Army  personnel)  of  varying  backgrounds  were  selected  for  the 
experiment.  Given  the  small  sample  size  (primarily  due  to  restrictions  in 
available  people,  time,  and  budget)  it  was  decided  to  dispense  with  detailed 
statistical  comparisons  and  simply  establish  general  conclusions  about  the 
repeatability  and  consistency  of  the  results.  The  replication  was  also  used 
to  develop  a  list  of  critical  factors  that  have  major  impacts  on  the  MPT 
projections . 

To  round  out  the  accuracy  examination  within  the  constraints  of  the 
study,  it  was  also  decided  to  perform  a  qualitative  accuracy  check,  using  past 
Air  Force  and  Navy  experience  with  the  same  type  of  methodology. 
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As  stated  earlier,  the  event-series  validity  test  is  extremely  important 
for  determining  the  required  accuracy  of  a  model  and,  subsequently,  the 
utility  of  a  new  simulation.  To  obtain  an  estimate  for  the  required  accuracy 
of  the  HCM,  a  simple  top-down  cost  sensitivity  analysis  was  employed  to 
establish  the  allowable  variance  in  the  manpower  projection,  as  constrained  by 
other  life-cycle  cost  elements  and  an  allowable  error  band  around  the  total 
life-cycle  cost  projection.  This  estimate  of  the  allowable  variance  in  the 
manpower  projection  was  performed  for  an  example  acquisition,  using  actual 
Army  weapon  system  data  (i.e.,  the  SINCGARS  system).  Once  the  level  of 
accuracy  was  established,  it  was  possible  to  formulate  corrective  actions  (for 
the  various  problem  areas  revealed  through  the  face-validity  and  internal 
reliability  investigations)  commensurate  with  a  greater  or  lesser  need  for 
accuracy.  It  was  for  this  reason  that  the  sensitivity  test  was  reserved  for 
the  end  of  the  evaluation.  The  complete  details  of  the  internal  validity  and 
accuracy  studies  are  provided  in  Section  VI. 


SECTION  IV 


USER  REQUIREMENTS  ANALYSIS 


A.  OVERVIEW 

As  a  part  of  Che  approach  developed  to  determine  the  face  validity  of 
the  HCM  MPT  projection  model,  an  MPT  user  requirements  study  was  conducted. 
This  analysis  included  a  definition  and  documentation  of  Army  user  require¬ 
ments  for  a  manpower,  personnel,  and  training  (MPT)  prediction  model.  Sub¬ 
sequently,  the  user  requirements  were  compared  with  the  capabilities  of  the 
HCM  to  determine  the  degree  to  which  the  technique  conformed  with  the  needs  of 
intermediate  as  well  as  end  users  of  MPT  information.  The  users  included 
those  who  currently  develop  the  MPT  information  as  well  as  those  who  primarily 
use  the  information  (Table  4-1).  The  details  of  the  requirements  analysis  are 
provided  in  the  following  paragraphs. 

B.  APPROACH 

This  study  focused  on  exploring  Army  user  requirements  for  a  "generic” 
MPT  methodology  that  would  serve  as  comparison  against  an  existing  MPT  pro¬ 
jection  technique  for  the  Army's  Integrated  Logistic  Support  (ILS)  process. 

The  user  requirements  provided  a  useful  measure  for  determining  the  extent  to 
which  the  HARDMAN  model  fulfilled  the  Army's  stated  requirements.  As  stated 
earlier  in  Section  III-D  and  III-E,  detailed  audits  and  a  test  case  replica¬ 
tion  were  also  conducted  to  assess  face  validity  and  internal  reliability. 

The  additional  familiarization  with  the  HCM  technique  achieved  through  these 
parallel  analyses  augmented  the  comparison  of  the  user  requirements  with  the 
methodology. 

The  basic  approach,  outlined  in  the  preceding  discussion,  is  explained 
in  greater  detail  in  the  following  paragraphs: 

1.  Selection  of  Individuals 

The  Interim  Report  for  Manpower  and  Personnel  Requirements 
Determination  Methodologies  (MANPERS)  states  that  "the  three  principal 
organizations  responsible  for  the  research,  development,  and  deployment  policy 
of  a  system  under  development  are  DARCOM  -  the  materiel  developer;  TRADOC  - 
the  training  and  doctrine  (i.e.,  combat)  developer;  and  HQDA  -  the  force 
modernization  planner.  With  the  exception  of  the  "industry"  user,  all  of  the 
users  interviewed  in  this  study  were  from  these  three  organizations  in  the 
Army.  These  commands  were  also  confirmed  by  the  sponsoring  agencies  (Army 
Research  Institute  (ARI)  and  the  Soldier  Support  Center  (SSC)),  as  key  actors 
in  the  Army's  MPT  process. 

A  predominance  of  users  were  from  the  TRADOC  community.  This  is  a 
reflection  of  the  fact  that  "front  end  analysis,"  such  as  MPT  projections, 
done  prior  to  Phase  I  in  the  systems  acquisition  process,  is  largely  a  TRADOC 
responsibility.  TRADOC  has  the  lead  in  ILS  management  prior  to  the  appoint¬ 
ment  of  a  DARCOM  Project  Manager  at  Milestone  I. 


Table  4-1.  User  Groups/Representatives  Interviewed 


Users 


Number  of 
Interviews 


ILS  Managers  (DARCOM)  2 
Soldier  Support  Center/SSC-NCR  (TRADOC)  3 
Logistics  Center/LOGCEN  (TRADOC)  1 
Deputy  Chief  of  Staff,  Combat  Development/DCSCD  (TRADOC)  1 
Deputy  Chief  of  Staff,  Training/DCST  (TRADOC)  2 
Special  Study  Group-TRADOC  Systems  MGR/SSG-TSM  (TRADOC)  1 
Logistics  Evaluation  Agency-DCSLOG/LEA  1 
DCSPER  1 
Industry  (based  on  results  of  survey)  1 


Total  User  Groups:  9  Total  Respondents3:  13 


aUsers  listed  were  selected  on  the  basis  of  their  role/function  in  the  MPT 
process.  All  users  contacted  participated. 


Inputs  from  the  SSC-NCR-II  (TRADOC)  and  industry  users  were  not  gathered 
from  actual  interviews.  The  information  needed  from  these  users  was  not 
obtainable  within  the  allocated  time  frame  for  the  study  and  was,  th'refore, 
extracted  from  documents  that  adequately  portrayed  their  respective  user  needs 
for  an  MPT  projection  model.  One  document  was  a  memorandum  about  requirements 
for  an  MPT  model  (9)  and  the  other  was  an  industry  survey  conducted  by  the 
Soldier  Support  Center  to  determine  industry  requirements  for  MPT  information 
in  developing  proposals  (10). 


2.  Questionnaire  Design 

The  questionnaire  was  designed  as  a  discussion  guide  to  identify 
user  requirements.  Some  questions  addressed  user  requirements  directly  by 
identifying  the  specific  MPT  information  input  they  require  to  fulfill  their 
responsibilities  for  the  Army  MPT  process.  Other  requirements  directly 
addressed  were  imbedded  in  questions  concerning  the  user's  interpretations  of 
the  timing  and  interfaces  required  by  the  process.  Through  interface  and 
timing  questions,  an  understanding  was  developed  about  when  critical  MPT 
information  is  needed  and  about  the  relationships  between  principal  organiza¬ 
tional  players,  so  that  it  could  be  determined  if  the  design  or  structuring  of 
an  MPT  projection  system  would  be  sensitive  to  these  relationships.  Another 
direct  way  of  focusing  on  user  requirements  was  to  ask  respondents  for 


.  ^WWOTSDBa*!*  JtiWlW.miLJBWlV 


suggestions  about  the  design  of  an  MPT  methodology.  Suggestions  included 
comments  on  outputs,  schedule,  and  the  process  of  actually  projecting  per¬ 
sonnel  needs.  Questions  indirectly  addressing  user  needs  provided  confirma¬ 
tion,  context,  or  further  elaboration  of  the  direct  user  data.  These  questions 
referred  to  the  following  contextual  information: 

(1)  The  user's  understanding  of  the  present  technique  used  for 
MPT  projections. 

(2)  Users  seen  as  responsible  for  MPT  projections  and  the  nature 
of  their  "use"  of  the  information  (intermediate  or  end  use). 

(3)  MPT  regulations /documents  that  impact  the  user. 

(4)  Users'  perceptions  of  the  strengths  or  weaknesses  of  the 
current  MPT  process. 

(5)  The  primary  charter  and  responsibilities  of  the  user. 

(6)  The  key  drivers  underlying  the  need  for  MPT  projections, 
according  to  each  user  interviewed. 

The  complete  questionnaire  is  shown  in  Appendix  A. 


3.  Interview  Procedure 

User  requirements  were  determined  primarily  through  interviews 
with  the  representative  users  who  were  identified  and  recommended  by  the 
sponsor.  Initial  contacts  made  by  ARI  provided  JPL  with  direct  access  to 
appropriate  individuals  with  whom  formal  follow-up  requests  for  assistance 
were  made  and  visits  scheduled. 

The  study  received  a  substantial  level  of  user  cooperation  in  conducting 
the  interviews;  an  example  was  that  users  frequently  called  upon  people  within 
their  organizations  having  specialized  knowledge  of  that  organization's 
involvement  in  the  MPT  process.  Each  scheduled  interview  was  treated  as  a 
single-user  interview  even  if  more  than  one  person  was  present  for  the 
discussion.  A  single-interview  form  consolidated  the  responses  of  all  present 
from  that  organization. 


C.  TABULATION  OF  INTERVIEWS 
1.  General  Responses 

Most  of  the  users  interviewed  thought  that  MPT  projections  should 
be  available  at  the  front  end  of  the  acquisition  cycle.  This  was  frequently 
linked  to  supportability  and  affordability  drivers  for  needing  MPT  information, 
particularly  because  these  affect  the  user  organization's  roles  and  responsi¬ 
bilities  for  MPT  projections.  One  user  cited  sustainability  as  a  key  driver 
of  MPT  information.  This  was  considered  an  important  insight  because  it 
highlights  life-cycle  aspects  of  system  operation.  The  predominant  responses 
pertaining  to  these  three  major  drivers  are  summarized  in  Table  4-2. 
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Table  4- 

■2.  Key  MPT  Drivers  and  Associated  User  Rationale 

Key  MPT  Drivers 

Predominant  Rationale 

Affordability 

MPT  system  must  provide  all  MPT  cost  elements  associated 
with  acquisitions 

MPT  system  must  reflect  cost  tradeoffs  with  force 
structure/mission  requirements 

Supportability 

MPT  system  must  support  readiness  goals 

MPT  system  must  fully  support  new  acquisitions 
(operations,  maintenance,  supply,  training,  etc.) 

MPT  system  must  demonstrate  sensitivity  to  horizontal 

MPT  impacts  (integrated  System  Support  concept) 

Sustainability 

MPT  system  must  support  manning/training  system(s) 
plans  over  long  periods  of  time 

MPT  system  must  consider  extended  impacts  of  mission/ 
scenario  changes  with  time 

Most  users  expressed  the  idea  that  system  design  should  be  influenced  by 
various  manpower  constraints  emerging  from  early  MPT  analysis  before  the 
design  is  fixed.  A  related  point  is  that  most  users  thought  that  the  primary 
tool  for  getting  MPT  projections  into  the  system  through  the  Quantitative  and 
Qualitative  Personnel  Requirements  Information  (QQPRI),  which  is  developed 
concurrently  with  the  Basis  of  Issue  Plan  (BOIP),  was  inadequate  and/or 
untimely.  By  the  time  QQPRI  data  is  available  (after  Milestone  I),  most  of  a 
system's  design  is  fixed  (Figure  4-1).  Another  problem  mentioned  was  that 
DARCOM  and  TRADOC 1 s  shared  responsibility  for  developing  the  QQPRI  and  BOIP 
requires  additional  clarification  of  lines  of  responsibility.  This  would  be 
particularly  important  if  the  initial  development  of  MPT  projections  shifts  to 
an  earlier  point  in  the  acquisition  cycle  (i.e.,  TRADOC  would  become 
responsible  for  the  initial,  therefore  tentative,  QQPRI). 

Occasionally,  a  user  expressed  a  viewpoint  that  MPT  projections  made 
later  in  the  acquisition  cycle  were  adequate,  providing  that  sufficient  lead 
time  was  given  to  acquire  and  train  the  personnel.  This  statement  assumed 
that  funding  was  already  available  for  the  personnel. 
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SOURCE:  Proceedings  from  National  Security  Industrial  Association 

Symposium  on  Navy  Systems  Acquisition,  October  27" 28,  1977- 


Figure  4-1.  Effect  of  Design  Decisions  on  Life-cycle  System  Cost 


2. 


User  Group  Responses 


User  responses  are  reviewed  in  this  section.  Responses  from 
multiple  users  in  DARCOM  and  SSC-NCR  are  incorporated  under  single  headings 
for  these  respective  organizations.  Individual  responses,  by  organization, 
are  detailed  in  Appendix  B.  The  discussion  on  user  responses  includes  a 
summary  of  the  user  organization  background  to  establish  the  appropriate 
context  for  the  responses  that  follow. 


a.  DARCOM/ Integra ted  Logistic  Support  (ILS)  Managers  Responses. 

ILS  managers  in  DARCOM  are  responsible  for  ensuring  the  planning  and  monitoring 
of  logistic  support  elements  for  systems  under  development  from  Milestone  I 
throughout  the  development  process.  The  prime  analytical  tool  used  in  ILS  is 
Logistic  Support  Analysis  (LSA). 

The  LSA  process  consists  of  analysis  efforts  to  completely  define  the 
support  system  and  its  interfaces  with  a  materiel  system.  The  analysis 
incorporates  manpower/logistics  analysis  and  tradeoff  analyses  associated  with 
maintenance  concepts  (repair  levels,  etc.).  LSA  Record  information  (LSAR) 
includes  all  tasks  required  to  operate  and  maintain  a  materiel  system.  LSA 
data  then  becomes  feeder  data  for  the  Quantitative  and  Qualitative  Personnel 
Requirements  Information  (QQPRI)  system.  As  the  chief  means  of  developing 
both  the  rumbers  and  kinds  of  personnel  required  for  operating  and  maintaining 
systems,  QQPRI  development  is  a  shared  responsibility  of  DARCOM  and  TRADOC. 

As  a  result,  LSA  data  needs  to  be  as  accurate  as  possible  for  both  the  DARCOM 
and  the  TRADOC  user  communities. 

LSA  normally  includes  operator/maintainer  task  and  supply  functions  for 
repair  and  test  equipment  as  well  as  the  system  under  development.  In  determin¬ 
ing  system  supportability ,  ILS  managers  required  information  about  the  exact 
tasks  to  operate  and  maintain  systems.  They  also  needed  projections  of  reli¬ 
able  manloading  requirements  to  help  establish  expected  Military  Occupational 
Specialty  (MOS)  and  skill  demands,  plus  any  MOS  changes  anticipated. 

ILS  managers  stated  a  need  for  methods  to  analyze  tradeoffs  between 
force  structure,  personnel  constraints,  hardware  performance,  and  cost. 
Essential  to  this  analysis  was  information  (output)  about  support  and  test 
equipment  ("playing  field")  associated  with  a  system.  The  total  QQPRI 
associated  with  an  expected  density  of  these  systems,  when  deployed,  was  cited 
as  an  important  element  of  this  analysis. 

Early  maintenance  concept  information  was  also  requested,  including 
maintenance  manhours  (MMH),  reliability,  availability  and  maintainability 
(RAM)  data,  and  maintenance-level  information. 

DARCOM  suggestions  for  the  design  of  an  MPT  model  included  providing/ 
displaying  MPT  data  for  alternative  (proposed),  as  well  as  predecessor  systems 
to  provide  support  documentation  for  decision-making.  An  MPT  methodology 
would  need  to  provide  credible  quantitative  data  to  the  ILS  process,  the 
source(s)  of  which  could  be  documented  or  traced.  Formats  for  the  data  should 
be  designed  to  present  data  efficiently  to  users  who  have  limited  time  for 
reviewing  information  (e.g.,  graphs,  charts,  etc).  Timing  for  initiating  the 
process  should  be  prior  to  Milestone  I  to  achieve  maximum  benefit  from 


hardware/manpower  tradeoffs;  however,  there  should  also  be  follow-up  on 
existing  systems  in  the  field  to  verify  LSAR  data  and  system  supportability 
information. 

b.  SSC-NCR  (TRADOC)  Responses.  SSC-NCR  is  responsible  for 
assessing  the  feasibility  of  allocating  personnel  and  materiel  to  specified 
organizations  within  the  Army  (i.e.,  the  Tables  of  Organization  and  Equipment 
(TOE).  SSC  also  ensures  that  the  user  communities  identify  both  demand  and 
supply  MPT  factors  at  the  front  end  of  the  system  acquisition  process  to 
address  supportability  and  affordability  issues. 

SSC-NCR  stated  that  detailed  data  for  the  following  general  categories 
of  information  is  required: 

(1)  Manpower. 

(2)  Comprehensive  task  breakdown. 

(3)  Personnel. 

(4)  Training. 

(5)  Logistic  support. 

(6)  Integrated  System  Support  (ISS). 

Graphic  display  capability  was  highlightd  as  useful  to  SSC  for  inclusion 
in  reports  comparing  demand  and  supply.  In  addition,  detailed  MPT  analyses 
comparing  proposed  to  predecessor  systems  is  required. 

SSC  also  needed  MPT  projections  prior  to  Milestone  I  to  support  the  Army 
Systems  Acquisition  Review  Council  (ASARC)  decision  process.  An  additional 
need  cited  was  timely  and  useful  cost  information  for  the  Army's  funding 
process.  In  providing  such  information,  the  projection  process  should  also 
expose  any  assumptions  employed  so  that  an  understanding  of  the  limitations  of 
the  analysis  is  clear. 

SSC-NCR  further  suggested  using  a  solid,  historical  data  base  containing 
Army  personnel  statistical  data,  comprehensive  task-related  files,  and  RAM 
data  associated  with  systems.  It  was  also  indicated  that  the  MPT  system 
should  be  compatible  with  existing  Army  reporting  structures  and  command  roles 
to  ensure  that  the  model  could  be  integrated  into  the  Army's  normal  systems 
acquisition  process. 

c.  Training  Developers  (TRADOC)  Responses.  A  key  concern  of 
training  developers  within  TRADOC  Headquarters  is  the  integration  of  the 
Army's  training  function  with  the  other  critical  Army  functions  -  manning, 
force  structure,  facilities,  equipping,  sustaining,  mobilizing,  and 
deploying.  Matching  the  Army  training  system  to  manning  and  the  systems 
acquisition  process  is  crucial  to  the  support  of  force  modernization. 
Supportability  of  force  modernization  requires  specific  information  about  the 
availability  of  manpower  qualified  for  training,  as  well  as  sustainability 
(personnel  flow  and  replacement)  of  the  force  required. 
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Another  driver  of  training  developers’  needs  for  HPT  information  was 
system  affordability,  i.e.,  an  understanding  of  a  system's  life-cycle  cost. 
The  general  categories  of  information  needed  by  training  developers  to 
determine  supportability  and  affordability  were  (1)  manpower  and  personnel 
information  (demand/supply);  (2)  training  information  (exact  operator/ 
maintainer  task  and  target  audiences);  (3)  equipment  information  (capability 
and  man/machine  tradeoffs);  and  (4)  accurate  maintenance  concept  information. 


d.  Combat  Developers/Soldier  Development  Directorate  (SDD)  and 
TRADOC)  Responses.  Combat  developers  in  TRADOC/ Soldier  Development 
Directorate  are  concerned  with  personnel  doctrine  that  addresses  air/land 
battle  mission  needs.  They  also  promote  combat  effectiveness  through  MOS  and 
force  structure  change  and  ensure  that  soldiers'  physical  and  mental  cap¬ 
abilities  are  taken  into  account  by  the  various  elements  of  force  moderniza¬ 
tion.  In  their  interface  between  SSC  and  TRADOC  Headquarters,  combat 
developers  at  SDD  are  involved  in  the  process  of  optimization  of  resources 
between  proponent  schools. 
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The  combat  developer's  key  drivers  for  needing  HPT  information  were 
s uppor tab i 1 i ty ,  in  a  readiness  sense;  performance ,  as  it  relates  to  combat 
effectiveness  (e.g.,  through  man/machine  design  issues);  and  cost  (e.g.,  as 
might  be  determined  by  impact  analysis). 


Combat  developers'  design  suggestions  focused  on  the  need  for  an 
interactive  management  information  system  having  a  time-line  signaling 
mechanism  that  could  alert  participants  to  needed  (or  missing)  actions  in  the 
current  HPT  process.  Suggested  outputs  from  the  HPT  model  were  a  projection 
of  the  numbers  and  kinds  of  operators  and  maintainers  associated  with  a 
system.  A  timely  breakdown  of  task  and  training  data  was  also  cited  to 
influence  system  design  (prior  to  Hilestone  I). 


It  was  also  suggested  that  some  information  concerning  the  impacts  of 
new  systems  on  the  existing  personnel  structure  and  resulting  tradeoffs  would 
be  useful  in  optimizing  the  selection  of  new  systems.  This  requirement  also 
implies  a  need  to  apply  HPT  analysis  to  systems  on  a  horizontal  plane  (i.e., 
to  consider  the  drain  on  the  projected  manpower  pool  caused  by  other  develop¬ 
ing  systems). 


e.  Tradoc  System  Hanagers  (TSM)  Responses.  During  the 
gestation  period  of  a  system  prior  to  Hilestone  I,  TRADOC  assumes  the  lead  in 
conducting  mission  area  studies.  A  TRADOC  Systems  Manager  (TSH)  is  concerned 
with  the  identification,  analysis,  and  selection  of  system  concepts  that  meet 
needs  of  combat  elements  of  the  Justification  of  System  Major  New  Starts 
(JSMNS)  document.  Other  responsibilities  of  the  TSM  include  preparing  the 
Operational  and  Organization  (0&0)  plan,  as  well  as  monitoring  the  preparation 
of  a  number  of  different  documents  required  for  logistic  support.  Among  these 
are  (1)  the  Integrated  Logistic  Support  Plan  (ILSP);  (2)  the  Concept  Formula¬ 
tion  Package  (CFP);  (3)  the  Outline  of  Individual  and  Collective  Training  Plan 
(OICTP);  (4)  the  Letter  of  Agreement  (LOA);  (5)  the  Cost  and  Operational  Effec¬ 
tive  Analysis  (COEA);  and  (6)  the  Preliminary  Qualitative  Assessment  (PQA).  The 


Preliminary  Quantitative  and  Qualitative  Personnel  Requirements  Information 
(PQQPRI)  could,  by  interpretation  of  the  current  regulation,  be  initiated  by 
the  TSM  although  it  is  not  normally  done  at  this  time. 

The  TSMs  had  a  clear  requirement  for  a  model  that  would  contain  a  com¬ 
prehensive  task  file  data  base  with  task  data  taken  from  a  number  of  task- 
centered  Army  manuals  and  documents.  This  would  assist  the  TSMs  in  ensuring 
the  timely  completion  of  these  documents  by  facilitating  the  identification  of 
specific  tasks,  MOS,  and  skills  associated  with  hardware.  Finally,  the  TSMs 
suggested  the  development  of  both  a  manpower  and  personnel  (demand/supply) 
data  base  that  would  permit  tradeoff  analysis  to  determine  the  best  fit  of 
people  with  equipment. 


f.  LOGCEN/LOGISTICS  CENTER  (TRADOC)  Responses.  Force 
developers,  training,  and  materiel  systems  managers  are  all  MPT  actors  within 
LOGCEN.  The  critical  role  of  LOGCEN  is  as  an  executive  agent  for  the  ILS 
process.  Issues  such  as  man/machine  interface,  system  and  non-system  training 
devices,  logistic  system  proponency,  and  Manpower  Authorization  Criteria 
(MACRIT)  are  included  in  LOGCEN' s  charter.  Some  of  the  LOGCEN  responsibil¬ 
ities  are  (1)  impact  analysis  of  system/design  characteristics;  (2)  support 
equipment  training  plan  and  technical  publications  development;  and  (3)  MACRIT 
system  improvement.  As  a  TRADOC  integrating  center,  LOGCEN  is  similar  to  a 
proponent  school  in  its  proponency  role  for  selected  logistic  systems.  LOGCEN 
also  develops  some  TOEs  for  maintenance  units. 

LOGCEN  indicated  they  had  specific  information  requirements  related  to 
system  considerations  for  affordability  and  supportability ,  including 
(1)  accurate  RAM  data  (based  on  historical  data  and  accounts  for  realistic 
utilization  rates,  actual  failure  rates,  and  combat  damage);  (2)  workload  data 
(as  a  function  of  direct  productive  and  indirect  productive  annual  maintenance 
manhours  (AMMH) ,  plus  available  productive  manhours);  (3)  density  of  equipment 
(TOE-related) ;  (4)  unit-focused  manpower  requirements;  and  (5)  LSAR-format 
task  data  associated  with  MOS  and  equipment  needed  for  training  analysis. 

LOGCEN' s  MPT  timing  requirements  were  twofold:  (1)  LOGCEN  needed  early 
(front-end)  training  device  information  to  prepare  for  both  early  system 
testing  and  the  final  system  design  configuration,  as  well  as  (2)  solid 
manpower  requirements  for  supporting  accurate  MOS  and  organizational  design 
decisions . 

Design  discussions  about  MPT  projection  models  focused  on  LOGCEN's 
requirement  for  accurate  and  reliable  (RAM)  data.  The  Annual  Maintenance 
Manhour  (AMMH)  data  that  LOGCEN  presently  receives  should  be  made  separate  and 
distinct  from  reliability  data.  MPT  information  was  needed  in  an  LSAR  data 
format  and  would  be  more  useful  if  broken  down  by  component,  task,  and  skill 
level.  It  was  felt  that  this  could  then  be  used  with  an  MPT  simulation  to 
determine  impacts  of  utilization  rate,  combat  damage,  and/or  hypothetical 
changes  in  RAM  characteristics. 

A  concern  was  expressed  about  the  accuracy  of  data  to  be  used  for  MPT 
analysis.  The  availability  of  "hard"  LSAR  data  is  critical  to  the  development 
of  useful  MPT  information.  Given  an  absence  of  system  fielding  experience  and 
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correspondingly  mature  LSAR  data  (such  as  would  be  the  case  when  substantial 
technological  changes  in  equipment  are  introduced),  it  was  recognized  that  it 
would  be  difficult  to  obtain  reliable  data.  However,  it  was  also  recognized 
that,  if  there  is  minimal  technological  change,  the  MPT  information  should  be 
readily  available. 


g.  Devices  and  Systems  Training  Directorate  (TRADOC)  Responses. 
This  organization  is  involved  in  policy  development,  training-device  pro- 
ponency,  and  training-plan  approval. 

In  general,  this  directorate  indicated  needs  for  both  manpower  and 
training  information  for  the  purpose  of  developing  the  individual  and 
collective  training  plans  (ICTP)  of  systems  under  development.  A  new  Army 
thrust  called  Integrated  System  Support  (ISS)  now  requires  the  generation  of 
MOS  information  that  is  system-  as  well  as  unit-specific.  It  was  also 
suggested  that  an  improvement  to  the  current  MPT  process  would  be  to  provide 
sufficient  front-end  task  manloading  and  training  information  to  allow  the 
design  to  be  influenced  early  in  the  acquisition  process.  One  final  expressed 
requirement  was  to  have  an  MPT  projection  model  that  more  effectively  flags 
MOS  "overload"  (i.e.,  a  model  that  detects  and  documents  additional  needed 
manpower,  both  numbers  and  skills). 


h.  Logistics  Evaluation  Agency  -  LEA  (DCSLOG)  Responses.  As  a 
field-operating  agency  of  the  DA  staff,  LEA  is  the  independent  logistician  in 
the  materiel  acquisition  process  responsible  for  monitoring  ILS  requirements 
for  systems  under  development.  Supportability  of  the  more  than  800  systems  to 
be  evaluated  was  cited  as  a  key  driver  for  needing  MPT  information,  along  with 
other  factors  such  as  degree  of  interoperability  and  performance  levels. 

LEA  users  indicated  a  need  to  be  able  to  use  MPT  projections  (before 
Milestone  I)  as  constraints  during  system  development.  These  MPT  projections 
should  include  RAM  goals ,  new  skills  and  training  device  concepts.  Following 
Milestone  I,  it  was  suggested  that  readiness  considerations  include  a  3-  to 
5-year  lead  time  to  acquire  and  prepare  logistic  support  personnel  prior  to 
system  fielding.  LEA  information  requirements  also  included  a  complete  and 
accurate  listing  of  tasks  and  skills  required  to  maintain  the  system 
(including  all  subsystems).  Finally,  there  was  an  expressed  requirement  to 
look  across  systems  (horizontal  analysis)  to  assess  overall  impact  on  logistic 
support  and  training. 


i .  Force  Modernization  Cell  -  Manpower  Programs  and  Budget 
(DCSPER)  Responses.  This  organization  provides  staff  support  to  DCSPER  for 
the  Army  Systems  Acquisition  Review  Council  (ASARC).  They  are  chiefly 
concerned  with  resource  management  for  force  modernization  and  bridging  demand 
and  supply  issues  in  both  manning  and  force-structuring.  DCSPER  is  concerned 
with  matching  the  number  and  kinds  of  "spaces"  needed  to  operate  and  maintain 
systems  with  the  required  qualified  personnel  (i.e.,  "faces").  In  this 
respect,  they  are  also  concerned  with  the  personnel  training.  Because  of 
congress ionally-mandated  "end  strengths"  for  the  Army,  information  concerning 
distribution  of  personnel  is  important.  Recognizing  that  affordability  and 


user  supportability  concerning  the  supply  of  personnel  is  a  realistic 
constraint,  DCSPER  therefore  suggested  that  a  horizontal  look  across  systems 
would  permit  analysis  of  the  impacts  of  shifts  in  the  numbers  and  types  of 
manpower  required.  Tradeoff  analysis  at  a  high  level  was  also  cited  as  useful 
for  conducting  Army-wide  optimization  studies  (e.g.,  hardware  vs.  manpower  vs. 
system  performance). 

Finally,  it  was  indicated  that  the  design  and  output  of  an  MPT  model 
should  be  compatible  with  existing  reporting  processes  (QQPRI,  BOIP,  ASARC 
Milestones,  etc.).  In  line  with  this  requirement,  it  was  also  suggested  that 
there  is  a  need  for  a  more  effective  (interactive)  information  system,  with 
appropriate  access  to  relevant  data  bases  of  MPT  information. 


j.  Industry  Responses.  Three  industrial  associations 
(Aerospace  Industries  Association,  Electronic  Industries  Association,  and 
National  Security  Industrial  Association)  were  surveyed  about  their  needs  for 
logistic  support  (MPT)  information  in  responding  to  Requests  for  Proposals 
(RFPs).  It  was  discovered  that  industry  is  very  interested  in  improving  its 
understanding  of  logistic  support  and  human  factors  concepts  associated  with 
systems  under  development.  Some  suggested  means  to  accomplish  this  were  (1) 
earlier  user  contact,  (2)  user  involvement  in  RFP  preparation,  (3)  better 
implementation  of  human  factors  analysis,  and  (4)  improved  information 
dissemination  to  industry.  Two  basic  concerns  of  industry  have  been  to 
address  human  factors  before  the  system  design  is  fixed  and  to  specify 
contract  arrangements  for  implementing  MPT  analysis. 

Industry  also  indicated  a  need  for  reliable  information  concerning  the 
inter ded/planned  concept  of  system  operation  and  supporting  organizational 
design.  The  participants  also  suggested  DOD  strengthen  its  estimates  of  the 
kinds  of  personnel  expected  to  operate  and  maintain  the  system.  Scenarios 
depicting  both  threat,  environment,  and  realistic  usage  rates  were  cited  as 
important  in  preparing  RFPs.  The  expected  performance  of  a  system,  plus 
maintenance  concept,  were  two  other  critical  data  points  contributing  to 
industry's  ability  to  conduct  or  contribute  to  early  MPT  analysis. 

The  industry-users  interviewed  also  indicated  a  need  for  MPT  information 
to  be  centrally  located  for  accessibility.  Charts  and  graphs  were  suggested 
as  being  especially  helpful  in  transmitting  to  industry  the  necessary 
information.  An  historical  data  base  was  also  suggested  to  permit  comparisons 
with  other  developed  systems  or  analysis  of  system  designs  that  were  rejected. 


2.  Tabulation  of  Individual  Interviews  into  Matrix  Format 

After  all  user  interviews  were  completed,  the  raw  discussion  notes 
were  assembled,  reviewed,  and  transcribed  onto  interview  forms  for  clarity. 

Key  words  and  ideas  were  highlighted.  Two  requirements  analysts  then 
separately  analyzed  the  information  and  jointly  tabulated  the  responses  to 
facilitate  analysis  of  the  results.  Once  the  answers  were  transformed  into 
rough  requirement  statements,  they  were  entered  into  a  matrix  format  to  permit 
a  clear  indication  of  which  users  were  associated  with  each  user  need.  The 
detailed  matrix  of  users  and  user  answers  is  shown  in  Appendix  B. 


D. 


TRANSLATION  TO  USER  REQUIREMENTS 


1.  Procedure 

The  user  needs  were  divided  into  two  categories  -  one  category 
containing  "output"  (data)  needs,  and  the  other  containing  "design"  needs  for 
an  MPT  projection  system. 

User  requirement  statements  were  generated  from  the  Appendix  B  matrix 
information  and  the  interview  forms.  When  appropriate,  similar  ideas  or 
requirements  expressed  by  different  users  were  translated  into  a  comprehensive 
user  requirement  statement . 

The  separate  requirement  statements  developed  by  both  analysts  were  then 
compared  to  ensure  agreement  and  completeness.  To  establish  priorities  among 
the  requirements,  criteria  were  developed  to  evaluate  the  requirements  in  the 
matrices.  A  final  user-requirements  list  was  developed  after  applying  these 
criteria. 


2.  Criteria  for  Assigning  Priorities 

The  following  criteria  were  considered  essential  to  determine 
which  requirements  were  necessary  or  desirable  for  an  MPT  projection  model: 

(1)  The  requirement's  role  in  addressing  key  drivers  associated 
with  MPT  projections  for  Army  systems  (i.e.,  affordability, 
supportability ,  and  sustainability). 

(2)  The  importance  of  the  requirement  in  providing  useful  and 
timely  MPT  information  to  the  Army  system  acquisition 
process . 

(3)  The  breadth  of  users  stating  the  requirement. 

3.  Requirements  List 

As  stated  earlier,  the  user  needs  were  categorized  by  "output"  and 
"design"  needs  and  then  prioritized  according  to  the  degree  of  fit  with  the 
three  criteria  stated  previously.  Once  this  procedure  was  completed,  the  user 
needs  were  grouped  into  mandatory  and  desirable  requirements.  Below  is  the 
final  list  of  user  requirements  (including  output  and  design  requirements)  for 
a  projection  system  that  conforms  to  the  needs  of  the  key  Army  players  in  the 
MPT  arena.  For  the  purpose  of  this  evaluation,  all  the  following  mandatory 
requirements  were  given  equal  weight  because,  ultimately,  they  were  primarily 
used  to  test  the  "reasonableness"  of  the  methodology: 

(1)  Mandatory  Output  Requirements: 

(a)  Operator/Maintainer  task  data  should  be  provided  as 
required  for  a  system  and  its  support  equipment 
(including  task  description  and  frequencies),  based  on 
a  replaced  or  comparable  system. 
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(b)  Maintenance  manhour  projections  must  be  based  on  a 
specified  maintenance  concept  (levels  of  maintenance, 
specific  RAM  data). 

(c)  Operation  and  maintenance  manpower,  personnel,  and 
training  (MPT)  projections  must  reflect  realistic 
system  usage  rates  as  determined  by  acquisition- 
specific  mission/scenario  drivers. 

(d)  Reasonably  accurate  (i.e.,  acceptable  to  the 
budget-control  community)  total  manpower  demand 
(number,  MOS,  skill  level,  additional  skill 
indicators)  is  required  before  Milestone  1. 

(e)  The  MPT  model  must  include  and  document  projected 
availability  of  personnel  for  the  system  (demographic 
data  such  as  number,  age,  sex,  size,  skills, 
education,  MOS,  term  of  service). 

(f)  The  MPT  model  should  document  MOS  alignment  data 
showing  current  status;  authorizations  vs. 
requirements  to  indicate  MOS  overloads. 

(g)  The  model  must  incorporate,  document,  and  use  target 
audience  description  (TAD).  This  includes  reading 
ability,  grade  levels,  experience,  skills,  and 
previous  training  of  available  personnel  to  determine 
accurately  testing/training  requirements  (the  TAD 
would  be  input  to  the  Letter  of  Agreement). 

(h)  The  "T"  of  MPT  must  include  the  formulation,  descrip¬ 

tion  and  use  of  comprehensive  training  concept 
elements:  (1)  classes  or  training  required  (including 

any  increases)  for  unit  or  institutional  training; 

(2)  time  to  train;  (3)  training  frequency;  and 
(4)  training  support  (devices,  facilities, 
instructors ) . 

(i)  There  must  be  documentation  of  MPT  impacts  and  costs 
on  the  Army  as  a  whole. 

(j)  Trade-offs  must  be  conducted  between  hardware  and 
people  to  determine  the  optimal  balance  for  best 
system  performance,  costs  (LCC),  and  support 
requirements  (Decision/Tradeoff  matrix  needed). 

(k)  Output  should  include  reports  with  well-labeled, 
easy-to-read  graphic  formats  such  as  graphs,  tables, 
matrices,  charts,  as  well  as  an  executive  summary  of 
the  report  that  could  stand  alone. 

(l)  Outputs  should  be  in  both  hardcopy  as  well  as  on  CRT 
display  for  permanent  archiving. 


(m)  Reporting  procedure  must  include  any  pertinent  caveats 
about  the  data  source  availability/reliability  or 
assumptions  about  the  system.  This  information  should 
be  provided  at  major  decision  points  or  steps  in  the 
analysis.  Appropriate  actions  to  resolve  identified 
weaknesses  in  the  analysis  should  also  be  noted. 

Desirable  Output  Requirements: 

(a)  A  complete  historical  MPT  data  base  is  highly  desir¬ 
able.  MPT  projections  should  be  updated  throughout 
the  life  of  the  system  to  increase  accuracy  of  LSA/ 
other  data. 

(b)  MPT  impacts  should  account  for  geographical  constraints 
on  manpower  pools  "available"  to  run  the  system. 

(c)  MPT  system  output  should  be  compatible  with  accepted 
LSA  reporting  format. 

Mandatory  Design  Requirements: 

(a)  Projected  MPT  impacts/costs  must  be  available  at  the 
front  end  of  the  acquisition  cycle  (before  Milestone 
1)  for  early  design  influence  and  early  MPT  (cost) 
warnings. 

(b)  Front-end  determination  of  a  system's  candidacy  for 
MPT  analysis  should  include  criteria  such  as  the 
potential  for  high  payoff  (e.g.,  selection  should  be 
based  on  type  and  amount  of  change  in  the  technology 
that  drive  significant  MPT  cos ts /changes ) . 

(c)  To  facilitate  training  analysis  and  to  support 
proponency  for  MOS  changes  or  additions,  the  system 
should  have  the  capability  to  compare  the  proposed 
operator  and  maintainer  tasks  associated  with  a  new 
system  with  a  comprehensive,  automated  data  base  of 
tasks  detailed  in  the  Soldier's  Manual,  ARTEP,  POI, 

SQT,  T.G.  and  AR  611-201  (Enlisted  Career  MOS 
Management  Fields). 

(d)  Output  should  include  comparable  MPT  projection  data 
for  both  alternative/proposed  and  existing  systems  as 
support  for  decision  milestones. 

(e)  The  model  should  incorporate  horizontal  analysis 
across  systems  to  determine  total  MPT  and  logistic 
impacts  and  to  facilitate  Army-wide  equipment  and 
personnel  optimization,  as  well  as  support 
end-strength  decisions. 

(f)  The  MPT  model  should  provide  compatability  with 
existing  DOD  documentation  and  reporting  structures. 

It  should,  however,  evolve  into  an  interactive 
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information  system  with  remote  terminals,  including  a 
time  line  oriented  to  both  the  budget  and  systems 
acquisition  processes.  Signalling  mechanisms  are 
needed  for  late/missing/incomplete  data. 

(g)  The  MPT  system  must  identify  and  provide  resolutions 
to  both  weaknesses  in  data  inputs  and  inaccuracies 
resulting  from  the  use  of  the  data. 

(h)  There  must  be  a  documented  and  accessible  audit  trail. 

(i)  A  universal  information  system  should  be  designed  to 
allow  multi-user  access  and  update. 

(4)  Desirable  Design  Requirements: 

A  more  detailed  0&0  plan  is  needed  to  act  as  a  "hingepin”  to 
initiate  MPT  analysis  at  the  front  end  of  the  acquisition 
cycle. 


E.  RELEVANCE  OF  USER  REQUIREMENTS  TO  EXISTING  HARDMAN  DESIGN 

Having  stated  the  user  requrements  in  the  preceding  section,  it  is  now 
possible  to  determine  the  extent  to  which  the  HCM  has  application  to  the 
requirements  for  the  Army's  existing  (or  near-term)  MPT  process.  Through  a 
comparison  of  the  HCM  to  user  requirements,  it  was  possible  to  establish  one 
measure  of  the  face  validity  of  HARDMAN  (i.e.,  how  well  the  HCM  fits  the 
Army's  MPT  needs).  This  was  achieved  by  checking  the  relative  applicability 
of  each  step  in  the  HCM  to  specific  portions  of  the  requirements.  As  stated 
earlier,  both  the  ARI  replication  exercise  and  the  HARDMAN  applications  audit 
were  conducted  in  parallel  with  the  requirements  analysis.  These  parallel 
studies  afforded  an  in-depth  understanding  of  each  step  in  the  methodology. 

The  additional  familiarization  with  the  complete  array  of  data  inputs, 
algorithms,  methodology  logic  process,  and  outputs,  provided  the  essential 
background  to  perform  a  comparison.  Table  4-3  presents  the  results  of  the 
compatibility  analysis  by  indicating  which  portions  of  the  HCM  have  high, 
medium,  or  low  applicability  to  the  user  requirements.  The  degree  of 
applicability  provided  an  indication  of  the  utility  of  various  portions  of  the 
technique  to  the  Army's  MPT  users. 

Out  of  the  thirteen  total  mandatory  output  requirements,  the  HCM 
demonstrated  good  compliance  in  essentially  six  of  the  user  need  areas.  Of 
the  seven  requirements  where  elements  of  the  methodology  appeared  to  exhibit 
low  applicability,  only  five  of  the  user  areas  were  considered  of  immediate 
importance  to  developing  a  sound  MPT  projection  scheme.  These  key  user  areas 
were  (1)  establishing  the  accuracy  of  the  MPT  methodology;  (2)  including  other 
demographic  data  such  as  age,  sex,  size,  etc.,  in  personnel  allocations; 

(3)  considering  the  demands  of  possible  future  acquisitions  on  MOS,  (4)  per¬ 
forming  a  complete  life-cycle  cost  analysis  as  part  of  the  tradeoff  studies; 
and  (5)  indicating  weaknesses  in  the  analysis  and  respective  resolution 
actions.  Items  (2)  and  (3)  appear  to  be  somewhat  of  an  Army-wide  constraint 
resulting  from  the  general  lack  of  readily  available  information.  Therefore, 
only  requirements  (1),  (4),  and  (5)  seem  to  be  achievable,  close-range 
deficiencies  that  should  be  incorporated  as  improvements  to  the  HCM. 


• 


Table  4-3.  Comparison  of  User  Requirements  to  the 
HARDMAN  Comparability  Methodology 


User  Requirements 


Applicability  of  HCM  Technique 
to  User  Requirements 


High  Medium  Low 


Manadatory  Output  Requirements 


Operator /maintainer  task  data  as  required 
for  system  and  support  equipment 
(description/frequencies)  based  on 
replaced  or  comparable  system 

X 

(Primary 

System) 

X 

(Support 

Equipment) 

Maintenance  manhour  projections  must  be 
based  on  specified  maintenance  concept 
(levels  of  maintenance,  accuracte  RAM 
data) 

X 

Operation/maintenance  requirements  must 
reflect  realistic  system  usage  rate  as 
determined  by  mission/scenario  drivers 

X 

Reasonably  accurate  total  manpower  demand 
estimate  (number,  MOS,  skill  level,  addi¬ 
tional  skill  indicators)  required  before 
Milestone  1 

X 

(Manpower 

demand) 

(Accuracy 

unknown) 

Model  must  include  and  document  projected 
availability  of  personnel  for  the  system 
(Demographic  data  such  as  -  number,  age, 
sex,  size,  skills,  education,  MOS,  terra 
of  service) 

X 

(No., 
skills  , 
MOS, 

Term  of 
service) 

X 

(Age/sex/ 

size 

education) 

Model  should  document  MOS  alignment  data 
showing  current  status;  authorizations 
vs.  requirements  to  indicate  MOS  overload 

X 

(Current 
system 
demands  ) 

X 

(Future , 
multi¬ 
system 
demands ) 

Model  must  incorporate  and  document  target 
audience  description  -  including  reading, 
grade  levels,  experience,  skills,  previous 
training  of  available  personnel  to  accu¬ 
rately  determine  tes ting/training  require¬ 
ments  (would  be  input  to  Letter  of 

Agreement ) 

X 

(Model 

assumes 

prede¬ 

cessor 

training 

is 

adequate 

Table  4-3.  (Cont'd.) 


User  Requirements 


Applicability  of  HCM  Technique 
to  User  Requirements 


High  Medium  Low 


Manadatory  Output  Requirements 


The  "T"  of  MPT  must  include  the  formula-  X 

tion,  description  and  utilization  of  (Insti- 

comprehens ive  training  concept  elements:  tutional 

classes  required  (including  any  increases)  train- 
for  individual;  unit  or  institutional  ing 

training:  time  to  train;  training 
frequency;  and  training  support  (devices, 
facilities,  instructors) 


? 

(Collec¬ 
tive  unit 
training) 


Documentation  of  MPT  impacts  and  associ¬ 
ated  costs  for  MPT  projection 

Tradeoffs  between  hardware  and  people 
needed  to  determine  optimal  balance 
between  performance:  costs  (LCC)  and 
support  requirements  (decision/tradeoff 
matrix  needed) 


X 


X 

(Limited 

tradeoff 

capability) 


X 

LCC  and 
other  O&S 
elements) 


Output  should  include  well-labeled,  easy- 
to-read  formats  such  as  graphs,  tables, 
matrices,  charts,  a6  well  as  an  Executive 
Summary  of  the  report  that  can  stand 
alone 


X 

( Executive 
Summary) 


Outputs  should  be  in  hardcopy  as  well  as 
CRT  display  for  permanent  archiving 


X  X 

(Hardcopy)  (CRT) 


Reporting  procedure  must  include  briefing 
re:  any  pertinent  caveats  about  the  data 
source  availability/reliability  or 
assumptions  about  the  system,  at  major 
decision  points  or  steps  in  the  analysis. 
Actions  should  be  noted  to  resolve 
weaknesses  identified 


X 

(Not  consis¬ 
tently  done) 


X 

(Resolu¬ 
tion 
actions 
on  major 
weak¬ 
nesses  ) 


Table  4-3.  (Cont'd.) 


User  Requirements 


Applicability  of  HCM  Technique 
to  User  Requirements 


High  Medium  Low 


I 

I 

i 

i 


r 


f 


Desirable  Output  Requirements 


Historical  MPT  database  highly  desirable. 
MPT  projections  should  be  updated  through¬ 
out  life  of  system  to  increase  accuracy  of 
LSA/other  data 


X 

(Histor¬ 

ical 

database) 


MPT  impacts  should  account  for  geo 
graphic  constraints  on  M.P.  pools  "avail¬ 
able"  to  run  the  system 

MPT  system  output  should  be  compatible 
with  accepted  LSA  reporting  format 


X 

(Update 

over 

system 

life) 

X 


X 


Mandatory  Design  Requirements 

Projected  MPT  impacts/costs  must  be  avail-  X 

able  at  the  front  end  of  acquisition  cycle 
(before  Milestone  I)  for  (a)  design 
influence  and  (b)  early  MPT  (cost) 
warnings 

Front-end  determination  of  system's  candi-  X 

dacy  for  MPT  analysis  should  include 
criteria  such  as  potential  for  high 
payoff,  e.g.,  selection  should  be  made  on 
type  and  amount  of  change  in  the  technol¬ 
ogy  that  mightttdrive  significant  MPT 
costs/changes 


To  facilitate  training  analysis  and  X 

proponency  for  MOS  changes  or  additions,  (New 
the  model  should  have  the  capability  to  skill 
compare  operator  and  maintainer  tasks  identifi- 

associated  with  a  selected  system  with  a  cation) 


comprehensive  automated  database  of  tasks 
detailed  in  the  Soldier's  Manual,  ARTEP, 
POI,  SQT,  T.E.  and  AR  611-201  (enlisted 
carreer-MOS  Management  Fileds) 


X 

(Auto¬ 

mation) 


User  Requirements 


Mandatory  Design  Requirements 

Output  should  include  comparable  MPT 
projection  data  for  alternative  (proposed) 
and  existing  systems  as  support  for 
decision  milestones 

Should  incorporate  horizontal  analysis 
across  systems  to  determine  total  MPT  and 
logistic  support  to  facilitate  ARMY-wide 
optimization  and  support  end  strength 
decisions 

MPT  model  should  provide  compatibility 
with  existing  DOD  documentation  and 
reporting  structures;  however,  it  should 
evolve  into  an  interactive  information 
system  with  remote  terminals,  including  a 
time  line  oriented  to  both  the  budget  and 
systems  acquisition  processes.  Signaling 
mechanisms  are  needed  for  late/missing/ 
incomplete  data 

MPT  system  must  identify/resolve  weak¬ 
nesses  in  data  inputs  and  resulting 
analysis 

Documented  and  accessible  audit  trail  is 
essential 

Universal  information  system  (CDB)  should 
be  included  and  designed  to  allow  multi¬ 
user  access  and  update 


Desirable  Design  Requirements 

More  detailed  0&0  Plan  required  as 
"hingepin"  to  initiate  MPT  analysis  at 
front  end 


Applicability  of  HCM  Technique 
to  User  Requirements 


High  Medium  Low 


X 


X 


X 

(Exist¬ 
ing  docu¬ 
menta¬ 
tion) 


X 

(Remote 

terminals 

with 

time-line 

overlay) 


X 


X 


X 

(Consol¬ 
idated 
data¬ 
base  ) 


X 

(User 
access ) 


X 

(Army 
respons i- 
bi lity) 


Out  of  the  nine  mandatory  design  requirements,  the  technique  seemed  to 
be  in  reasonable  agreement  with  seven  of  the  user  needs.  The  two  areas  where 
there  was  low  agreement  were  considered  critical  for  the  near  term.  These  two 
areas  were,  (1)  consideration  of  the  horizontal  effects  of  other  systems  on 
the  manpower  drain  and  (2)  construction  of  a  means  to  flag  and  resolve 
weaknesses  in  the  MPT  analysis.  Both  Items  (1)  and  (2)  were  already  discussed 
as  being  an  equally  important  consideration  on  the  "output"  side  of  the  system 

Overall,  this  discussion  suggests  that  the  HCM  is  reasonably  applicable 
to  the  Army's  major  user  needs  in  its  present  design.  More  specifically,  the 
present  HCM  basically  meets  seventeen  of  the  twenty-two  crucial  user  needs. 
Therefore,  by  being  largely  commensurate  with  the  present  user  requirements, 
the  technique  seems  to  have  considerable  near-term  utility  to  the  Army. 


SECTION  V 


OPERATIONAL  ANALYSIS 


A.  OVERVIEW 

Section  IV  provided  one  measure  of  face  validity  by  indicating  the 
relative  compliance  of  the  HCM  with  independently-derived  user  requirements. 
This  part  of  the  analysis  suggested  that  the  methodology  had  considerable 
utility  to  the  Army.  Other  important  face-validity  considerations  were 
introduced  in  Section  III.  These  considerations  refer  more  to  the  general 
integrity  of  the  technique  and  include  such  elements  as  potential  error 
sources,  logic,  reasonableness  or  credibility  of  the  projected  results, 
inherent  ambiguities  in  the  logic  or  outcome,  and  how  supportable  the  model  is 
by  its  respective  input  data  sources.  To  close  out  these  last  elements  of  the 
face-validity  analysis,  a  multi-leveled  approach  was  structured,  basically 
consisting  of  (1)  a  comparison  of  the  HCM  against  any  MPT  modeling  schemes 
delineated  in  existing  DOD  and  MIL-STD  instructions;  (2)  a  comparison  of  the 
HCM  against  a  host  of  other  known  MPT  projection  or  modeling  schemes;  and 
(3)  an  independent,  detailed,  step-by-step  audit  of  actual  applications  of  the 
technique.  This  part  of  the  overall  study  was  termed  "operational  analysis" 
because  every  aspect  of  the  methodology  was  actually  exercised  quite 
rigorously.  The  following  paragraphs  first  provide  the  detailed  rationale  and 
approach  used  to  investigate  the  various  remaining  face  validity  issues, 
followed  by  the  actual  analysis  and  conclusions  concerning  the  "general" 
integrity  of  the  HARDMAN  system. 


B.  APPROACH 

To  ensure  that  the  HCM  was  generally  sound,  beyond  the  user  requirements 
compliance,  several  additional  face-validity  elements  were  considered.  The 
additional  face-validity  elements  and  respective  tests  were  as  follows: 

(1)  Logic  -  A  simple  examination  of  whether  the  proposed  technique 
maintains  a  well-structured,  credible  train  of  logic  throughout 
the  model. 

(2)  Usability  -  A  simple  study  of  both  the  utility  of  the  model  and 
the  ease  with  which  the  model  can  be  applied  by  others. 

(3)  Error  Sources  -  An  investigation  of  whether  the  methodology 
incorporates  data  inputs  or  internal  procedures  that  can  cause 
inaccuracies  in  the  projections. 

(4)  Data  Availability  -  A  basic  check  on  whether  the  modeling 
technique  draws  on  real-world,  available  data  sources  to  act  as  a 
foundation  for  the  projections. 


(5)  Ambiguity  -  An  examination  of  the  ability  of  the  model  to  be  both 
consistent  and  succinct  in  its  respective  use  of  data  and  overall 
design  (e.g.,  having  each  successive  step  build  on  previous 
steps).  The  ambiguity  test  also  includes  a  look  at  how  cl.early 
the  results  relate  to  the  model's  original  objective(s). 

(6)  Credibility  -  A  simple  test  of  whether  the  results  seem 
reasonable,  based  on  the  inputs  and  whether  the  technique 
incorporates  a  means  of  weighing  the  quality  of  the  data  inputs 
and  results. 

A  three-phased  analysis  was  structured  within  the  context  of  the 
preceding  face-validity  tests.  The  first  and  second  phases  of  the  analysis 
primarily  addressed  the  logic,  usability,  error  source,  and  data-availability 
elements.  The  third  phase  of  the  face-validity  analysis  encompassed  all  six 
of  the  above  elements. 

As  an  initial  step  in  evaluating  any  new  modeling  scheme,  it  is  useful 
to  compare  the  new  methodology  against  similar  techniques  that  appear 
reasonable  and/or  are  already  tested  and  in  use.  This  comparison  serves 
several  purposes  in  the  context  of  the  above  first  four  validity  elements. 
First,  it  provides  a  basis  to  examine  whether  the  overall  logic  and  structure 
of  the  model  are  reasonable.  Second,  the  comparison  establishes  a  yardstick 
against  which  to  measure  improvements  in  the  versatility  and  subsequent 
utility  of  the  model.  Third,  it  assists  in  flushing  out  inherent  sources  of 
errors.  Last,  the  comparison  sheds  light  on  the  availability  and  quality  of 
data . 

To  these  ends,  the  first  phase  of  the  validity  analysis  examined 
existing  DOD  and  MIL-STD  guidelines  to  see  if  any  credible  MPT  projection 
schemes  have  already  been  established.  Similarly,  the  second  phase  consisted 
of  conducting  a  much  larger  literature  search  inclusive  of  past  and  present 
MPT  models  from  both  the  government  and  private  sectors. 

The  third  phase  of  the  face  validity  study  was  a  detailed  audit  of  three 
applications  of  the  projection  technique.  This  part  of  the  operational 
analysis  proved  to  be  a  pivotal  component  because  it  encompassed  all  the 
validity  insights  provided  by  the  above  literature  searches,  as  well  as  a 
grasp  of  (1)  the  ease  with  which  the  model  could  be  applied  by  others;  (2) 
whether  the  technique  contained  discontinuities  between  its  inputs,  internal 
mechanics  and  results  (therefore  introducing  ambiguities);  and  (3)  whether 
there  was  a  clear  indication  of  the  quality  and  limitation  of  the  outcomes. 

The  three  applications  chosen  for  the  audit  were  the  Single  Channel  Ground 
Airborne  Radio  System  (SltfCGARS) ,  Remotely  Piloted  Vehicle  (RPV),  and  the 
Division  Support  Weapons  System  (DSWS).  The  results  of  the  three  remaining 
phases  of  the  face-validity  analysis  are  detailed  in  the  following  paragraphs. 
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C.  ANALYSIS  OF  EXISTING  DOD  MPT  PROJECTION  GUIDELINES 

As  indicated  in  Section  II,  there  are  several  DOD  and  MIL-STD  documents 
that  refer  to  MPT.  For  example,  DOD  5000.1  and  5000.2  describe  the  structure 
and  supporting  procedures  for  planning  and  completing  major  system  acquisi¬ 
tions  (1,2).  As  a  precursor  to  5000.2,  the  5000.1  directive  calls  out  the 
necessity  to  define,  estimate,  and  budget  accordingly,  MPT  at  the  acquisition 
management  level.  The  5000.2  directive,  acting  in  a  supplementary  role,  more 
specifically  addresses  how  the  new  acquisition  will  be  evaluated  against  the 
various  support  parameters  such  as  MPT.  For  example,  using  a  top-down 
approach,  it  first  defines  the  acquisition  review  council  (DSARC  or  ASARC) 
advisory  roles  and  then  establishes  both  the  content  and  approximate  time 
frames  for  the  major  acquisition  and  system  development  review  phases.  From 
the  standpoint  of  threat  mitigation,  affordability,  and  supportability , 
detailed  guidelines  are  provided  for  developing  the  system  acquisition  and 
support  strategy.  The  5000.2  directive  states  the  necessity  to  consider  MPT 
needs  under  the  "System  Readiness,  Support,  and  Personnel  (ILS)"  portion  of 
the  document.  Although  this  subsection  requires  that  MPT  estimates  consider 
both  peacetime  and  wartime  manpower  demands,  skills,  costs,  and  training,  it 
does  not  indicate  a  methodology  to  assist  in  making  these  estimates. 

Similarly,  the  5000.1  directive  provides  even  less  distinct  guidelines  for 
determining,  and  budgeting  for,  MPT  costs.  Because  MPT  falls  under  the 
heading  of  Integrated  Logistics  Support  (ILS),  the  next  step  was  to  examine 
directives  more  specifically  related  to  system  support.  Perhaps  one  of  the 
best  of  the  family  of  directives  associated  with  system  support  is  MIL-STD 
1388A  (11).  This  MIL-STD  states  up  front  the  requirement  to  develop  human 
engineering  and  safety  programs  as  part  of  the  overall  system  support  analysis 
plan.  The  document  then  proceeds  to  delineate  key  MPT  variables  and  sup¬ 
porting  data  that  must  be  considered  to  determine  whether  the  proposed  system 
meets  manpower  readiness,  supportability,  and  affordability  goals.  These 
variables  include  (1)  the  number  of  systems  to  be  supported;  (2)  missions  per 
unit  of  time;  (3)  mission  duration;  (4)  operating  parameters  (days,  miles, 
hours,  flights,  etc.);  (5)  maintenance  and  transportation  periods;  (6)  des¬ 
criptions  of  system  components  and  operating  environment  (in  terms  of  their 
impact  on  MPT);  (7)  task  descriptions;  (8)  costs;  and  (9)  special  support  and 
test  equipment  MPT  impacts.  Additionally,  MIL-STD  1388A  requires  that  all  the 
various  support  scenarios  for  each  proposed  design  alternative  be  compared 
against  each  other  to  optimize  the  design  selection. 

As  in  the  case  of  the  previously  described  DOD  directives,  MIL-STD  1388 
also  fell  short  of  providing  an  analytic  structure  for  projecting  new 
acquisition  MPT  demands.  Although  a  dead-end  from  the  analytic  standpoint, 
MIL-STD  1388A  did  provide  more  depth  on  some  positive  characteristics  of  the 
HCM.  These  characteristics  revolved  around  the  reasonably  close  match  between 
the  DC®  MPT  guidelines  and  structure  of  the  data  inputs  and  outputs  of  the 
HCM.  Because  MIL-STD  1388  provided  more  specific  guidelines  on  MPT  support 
considerations  than  DOD  5000.1  and  5000.2,  it  was  decided  to  probe  one  level 
deeper  into  specific  Army  directives  having  to  do  with  MPT.  At  this  final 
level  it  was  anticipated  that  all  key  MPT  variables  and  data  elements  would  be 
defined  along  with  the  supporting  analytic  structure.  Probably  the  most 
concise  Army  directive  in  the  MPT  area  is  Army  Regulation  (AR)  570-2  (12). 
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This  service  regulation,  entitled  the  "TOE  Manpower  Authorization  Criteria 
(MACRIT),"  provides  criteria  and  standards  that  help  establish  an  "equitable 
relationship  between  services  performed  and  types  of  personnel  utilized."  To 
achieve  this  match  the  MACRIT  requires  that  system  manpower  studies  be 
conducted  and  the  results  approved  by  the  Deputy  Chief  of  Staff  for  Personnel 
(DCSPER)  for  the  Army.  The  studies  must  provide  the  number  of  direct  workers 
necessary  to  effectively  perform  a  specified  work  activity.  Direct  (actual 
task  performance),  nonproductive  (sleep),  and  indirect  (travel  and  task 
preparation)  times  must  be  considered  in  the  manpower  calculation.  The  MACRIT 
defines  some  approximate  time  factors  for  these  three  elements  and  ultimately 
provides  a  simple  equation  to  calculate  manpower.  This  equation  is  as  follows 
(12): 


WORKLOAD  FORCE  STRUCTURE 

(in  direct  and  X  (in  number  of 

indirect  manhours  per  system) _ systems  to  be  procured) 

AVAILABLE  PRODUCTIVE  HOURS 


as  a  rectangular  function. 
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One  interesting  aspect  of  the  MACRIT  manpower  equation  is  that  the 
number  of  support  people  is  calculated  for  the  complete  procurement  totally 
independent  of  the  acquisition  time  cycle.  Therefore,  if  the  manpower  require¬ 
ment  per  year  was  graphed  as  a  function  of  the  system  life  cycle,  one  would 
observe  the  same  type  of  step  function  as  shown  in  Figure  5-1,  indicating  no 
change  in  the  manpower  requirements  over  time.  This  function  is  discussed  in 
greater  detail  later  in  this  section.  The  most  important  aspect  of  the  above 
discussion  is  that  AR570-2  presently  represents  the  key  manpower  projection 
and  planning  document  in  the  Army.  It  is  now  also  obvious,  based  on  the  HCM 
summary  provided  in  Section  II-B,  that  the  methodology  is  strongly  aligned 
with  the  manpower  guidelines  as  specified  in  the  MACRIT  as  well  as  the 
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broader  policy  guidelines  spelled  out  in  major  support  and  acquisition 
planning  documents  such  as  MLL-STD  1388A,  DOD  5000.1  and  DOD  5000.2. 

It  is  clear  from  this  discussion  that  related  DOD  directives  do  not  seem 
to  provide  much  analytic  guidance  in  the  area  of  new  acquisition  MPT  projec¬ 
tions.  Although  one  appraisal  element  was  obtained  via  the  MACRIT,  this  data 
point  only  represented  one  aspect  of  evaluating  a  much  more  complicated  metho¬ 
dology  encompassing  task  and  skill  analysis,  personnel  identification,  training 
identification,  costing,  and  tradeoff  analysis.  To  obtain  more  data  points  on 
the  manpower  algorithm  and  evaluate  the  other  methodology  elements,  it  was 
obvious  that  a  broader  literature  search  was  required.  The  following  section 
provides  the  results  of  this  broader  effort. 


D.  CRITIQUE  OF  OTHER  EXAMPLE  MPT  PROJECTION  METHODS 

The  summary  of  directives  provided  in  the  previous  discussion  was  not 
meant  to  be  an  exhaustive  review  of  all  related  MPT  DOD  documentation.  For 
the  purpose  of  this  study  it  sufficed  to  point  out  that  the  immediate  and  most 
widely  used  DOD  acquisition  and  support  guidelines  are  not  fruitful  sources  of 
detailed  MPT  modeling  techniques.  A  somewhat  similar  approach  was  taken  with 
the  literture  search.  Again,  the  intent  was  not  to  cover  all  modeling 
developments  in  MPT.  For  this  study  it  was  adequate  to  examine  a  sufficient 
number  of  related  papers  and  models  to  obtain  a  solid  understanding  of  other 
accepted  techniques  and  logic  employed  in  government  and  the  private  sector, 
compare  shortcomings  of  both  the  DRC  and  other  models,  and  ensure  that  a  large 
enough  array  of  models  was  examined  so  that  all  aspects  of  the  HARDMAN 
methodology  could  be  evaluated.  This  approach  turned  out  to  be  very  efficient 
because  the  literature  search  revealed  a  considerable  amount  of  redundancy  in 
the  MPT  material.  In  line  with  this  finding,  the  following  sumnary  and 
analysis  encompass  a  select  group  of  papers  that  appeared  to  have  the  best 
data  and  analytic  foundations  as  related  to  their  respective  aspect  of  the  MPT 
problems.  As  stated  earlier,  the  HOI  covers  a  broad  number  of  MPT  topics. 
Therefore,  to  prevent  any  confusion  in  understanding  how  the  various  papers 
apply  to  the  different  topics,  the  following  discussion  is  organized  by  each 
step  in  the  methodology.  The  methodology  steps  described  earlier  in  Section 
II-D  are  restated  here  in  a  little  more  depth  to  assist  the  reader  in  seeing 
the  distinctions,  or  parallels,  between  the  different  projection  techniques. 


1.  Summary  of  Papers  Related  to  Building  the  HCM  Historical  Data  Base 
(Step  1) 

Step  1  of  the  HOI  requires  the  development  of  a  reference  system 
that  is  representative  of  the  proposed  system  (meets  the  functional  require¬ 
ments)  and  is  generally  composed  of  technically  and  operationally  similar 
predecessor  components.  The  development  of  the  reference  system  is  a  pivotal 
point  in  the  MPT  analysis  because  a  conceptual  error  in  system  configuration 
could  result  in  gross  MPT  errors  later  in  the  analysis.  The  premise  for 
conceptually  building  the  proposed  system  out  of  predecessor  components  is 
based  on  the  idea  that  technology  growth  is  usually  evolutionary  rather  than 
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revolutionary  in  nature.  Two  references  were  discovered,  Fong's  unmanned 
spacecraft  cost  model  and  Gordon's  review  of  mining  equipment  R&D,  that 
illustrate  the  "evolutionary"  concept  of  technology  development  as  being  a 
reasonable  and  supportable  assumption  (13,14).  Both  the  Fong  and  Gordon 
papers  were  chosen  to  show  that  the  accepted  concept  of  evolutionary 
technology  development  in  fact  spans  several  industries  (i.e.,  the 
aerospace/communications  industry  to  the  mining  industry). 


2.  Summary  of  Information  Related  to  the  Manpower  Algorithm  (Step  2) 

As  stated  earlier  in  this  section,  one  type  of  manpower  projection 
algorithm  is  the  simple  Army  MACRIT  form  (12).  This  form  is  the  one  employed 
in  Step  2  of  the  HCM  where  the  mission,  operational,  and  maintenance  data 
(extracted  from  the  proposed  and  adjusted  predecessor  system  data  bases),  are 
combined  to  establish  the  new  system  manpower  requirements.  Quade  and 
Boucher,  in  their  book  entitled  Systems  Analysis  and  Policy  Planning,  employ 
the  same  basic  rectangular  function  for  projecting  weapon  system  manpower 
support  (15).  They  suggest  that  this  algorithm  is  desirable  largely  because 
it  provides  a  simple  approximation  of  manpower  (15).  Another  interesting 
insight,  somewhat  in  conflict  with  the  Quade  and  Boucher  approach,  is  provided 
by  Putnam  in  his  paper  dealing  with  projecting  manpower  to  operate  a  new 
computer  software  system  (16).  Putnam  provides  essential  reference  backup 
supporting  the  theory  that  raedium-to-lar ge  sized  systems  follow  a  life-cycle 
pattern  of  (1)  rise  in  manpower,  (2)  peaking,  and  (3)  gradual  tailing  off  as 
the  system  gets  older  and  is  attrited.  This  cycle  is  termed  the  Norden/ 
Raleigh  cycle  and  closely  approximates  a  critically  damped  oscillating  system 
or  a  Weibull  curve  as  shown  in  Figure  5-2. 


t  =  LIFE  CYCLE  IN  MONTHS  OR  YEARS 


Figure  5-2.  Typical  Norden/Raleigh  Manpower  Curve 


The  equation,  depending  on  the  boundary  conditions  and  how  gradually  the  curve 
peaks  and  tapers  off,  is  of  the  form: 

2 

MP  «  Cx+  C2t  e“bt,  or,  MP  =  C2t  e"bt 


where  Cj,  C2,  and  b  are  constants  determined  by  the  boundary  conditions 
MP  *  0  @  t  *  0,  dMP/dt  »0@t“tp(t  peak),  MP  =  MPmax  @  t  ■  tp 
(t  peak).  The  total  manpower  required  for  the  life  of  the  system  is 
determined  by  integrating  the  above  funtion(s)  over  the  total  life  cycle.  The 
important  thrust  of  this  paper  is  not  determining  the  exact  MP  equation,  but 
the  fact  that  the  author  provides  substantial  backup  indicating  that  small, 
short-lived  projects  can  be  modeled  fairly  accurately,  using  a  rectangular  MP 
function  (16).  However,  large,  long-lived  programs  (such  as  a  large-weapon 
system)  may  require  a  manpower  curve  reflecting  the  more  accurate  buildup  and 
tapering  off  of  real-life  systems  (16).  This  could  have  major  impacts  on 
early-  or  later-stage,  life-cycle  MP  requirements  because  the  rectangular 
function  would  show  a  larger  requirement  than  actually  needed.  Consequently, 
a  new  system  being  jointly  phased-in  at  the  same  time  or  at  a  later  stage 
might  not  have  access  to  a  needed  manpower  pool  if  the  rectangular  curve  were 
used  as  an  acquisition  planning  guide. 

3.  Summary  of  Papers  Related  to  the  HOI  Training  Resource  Assessment 

(Step  3) 

Step  3  of  the  HOI  addresses  the  determination  of  training 
resources.  Step  2  of  the  procedure  provides  both  the  number  and  types 
(skills)  of  people  required  for  the  proposed  systems.  Drawing  on  this 
information,  simple  programnatic  matchups  based  on  existing  schools  and 
training  algorithms  are  employed  to  calculate  the  kinds  and  number  of  courses, 
instructors,  training  devices,  and  total  training  costs.  The  approach  uses 
present  curricula,  devices,  and  associated  costs  where  predecessor  system 
components,  similar  to  the  proposed  system,  already  exist.  However,  where  new 
skills  and  training  may  be  required  the  methodology  provides  logic  algorithms 
(in  the  form  of  decision  trees)  for  (1)  determining  whether  additional 
training  is  required  and  (2)  whether  (and  what  kinds  of)  generic  training 
devices  are  required.  Ballpark  cost  estimates  are  then  developed  by  using 
present  costs  of  the  generic  devices  as  they  exist  in  the  other  services  or 
industry . 

The  PACER  document,  introduced  earlier  in  Section  II,  is  almost 
uniformly  in  consonance  with  the  HCM  (7).  One  effort  derived  from  the  HCM  and 
developed  by  Winston,  "A  Preliminary  Report  on  the  Early  Comparability 
Analysis  (ECA)  Methodology,"  examines  the  difficult  problem  of  identifying, 
focusing  in  on,  and  training  for,  new  hard-to-find  skills  (17). 
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Winston's  effort  primarily  addresses  the  issues  of  force  modernization 
versus  the  recrui tment/ retention  of  manpower.  The  object  of  the  comparability 
analysis  is  to  identify  potentially  high  HPT  drivers  early  in  the  acquisition 
process.  Data  on  specific  problem  tasks  or  "high  drivers"  would  be  supported 
by  historical,  current,  or  projected  data  on  MPT.  Examples  of  data  sources 
include  (1)  Army  Occupational  Survey  Program  (AOSP),  (2)  proponent  schools 
(TRADOC),  (3)  subject  matter  experts,  (4)  Logistic  Support  Analysis  Records 
(LSAR),  and  (5)  Amy  Publications.  The  basic  methodology  involves  the 
following  steps: 

(1)  List  the  required  tasks  and  MOS  for  the  proposed  system  by  using  a 
similar  predecessor. 

(2)  Establish  task  criteria  such  as  %  of  people  performing  a  task 
(PPT),  task  difficulty  (TD),  frequency  of  task  performance  (FTP), 
risk  involved  if  task  not  performed  properly  (RI) ,  task  learning 
difficulty  (TLD),  training  time  (TT),  and  task  decay  rate  (TDR). 

(3)  Assign  subjective  values  of  1-4  (where  1  is  "low"  and  4  is  "high") 
to  each  criterion,  for  every  task  identified  with  the  proposed 
system. 

(4)  Calculate  a  subjective  task  score  for  each  task  (i.e.,  PPT  x  TD  x 
FTP  x  RI  x  TLD  x  TT  x  TDR  *  Task  Score). 

(5)  Identify  high  drivers. 

(6)  Conduct  task  analysis  of  high  drivers. 

(7)  Establish  learning/ training  requirements  for  high  drivers. 

(8)  Determine  personnel  required  based  on  test  scores  and  available 
skills. 

The  logic  algorithms  employed  in  the  HCM  decision  tree  analysis  of  new  skills 
use  the  same  subjective  task  criteria  as  listed  in  Step  2  of  the  preceding 
comparability  methodology. 


4.  Summary  of  Papers  Related  to  the  HCM  Personnel  Requirements 
Analysis  (Step  4) 

Step  4  of  the  HCM  establishes  the  necessary  number  of  people  in 
the  training  pipeline  to  support  the  manpower  and  skill  projections  made  in 
Step  2.  The  pipeline  figure  considers  that  personnel  move  in  and  out  of 
various  skill  categories  for  reasons  such  as  retirement,  promotion,  or 
changing  to  different  skill  areas.  The  fractional  change  in  people  moving  in 
and  out  of  various  skill  areas  is  based  on  historical  data. 
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Several  papers  associated  with  modeling  this  flow  process  were  found. 
Driven  by  a  desired  manpower  ceiling  for  various  skills,  each  model  basically 
establishes  the  monthly  or  yearly  proportional  ebb  and  flow  of  people  within 
each  skill  category  (based  on  historical  data)  and  compares  the  final  remain¬ 
ing  stock  of  people  in  each  skill  area  against  the  desired  ceiling  to  estab¬ 
lish  manpower  shortages.  Papers  by  Enke  (18),  Holz  (19),  Safeer  (20),  and 
Doong  (21),  apply  specifically  to  the  military  (i.e.,  Army  and  Navy);  while 
papers  by  Drummond  (22),  and  Leeson  (23),  examine  the  problems  of  statewide 
agricultural  labor  shortages  and  police  shortages,  respectively.  Although  the 
terms  in  the  equations  describing  the  personnel  flow  process  vary  from  paper 
to  paper,  the  general  algorithms  are  the  same.  The  algorithms  have  the 
following  basic  form: 


n  n 

V  (MP  ) . =  V  P.  (1  -  (f  -  fTM  +  f_)). 
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where 

(MP^).  =*  Manpower  available  in  the  ith  skill  category  after  reducing 

R  *  the  existing  pool 

P^  =  Existing  or  prjected  pool  of  people  in  the  ith  skill  category 

f-u  *  Fraction  of  people  out-migrating  from  skill  category  i 
UM 

fTU  =  Fraction  of  people  in-migrating 
IN 

f  =  Fraction  of  people  retiring 
K 

(MP  ) .  =  Desired  manpower  ceiling  for  the  ith  skill  category 

w  X 

MP£  *  Total  entrant  manpower  required  to  sustain  the  desired 
manpower  ceiling 

Enke  suggests  using  probabilities  in  place  of  the  fractional  adjustments  of 
the  manpower  pool  to  determine  the  possibility  of  joint  events  occurring 
(e.g.,  the  chances  of  an  individual  leaving  the  Army  within  the  interval  t  ♦  n 
given  that  he  has  attained  a  certain  level  of  training  in  a  given  skill  area) 
(18).  Enke  further  indicates  that  variables  such  as  the  state  of  the  economy, 
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whether  there  is  or  is  not  a  draft,  or  massive  reorganizations  can  effect  the 
personnel  flux  greatly  (18).  Of  all  the  papers,  the  most  satisfactory  approach 
to  dealing  with  these  variables  is  provided  by  Enke.  Enke  points  out  that 
historical  manpower  flows  are  only  useful  if  (1)  stable  socio-economic  condi¬ 
tions  have  existed  and  will  continue  to  exist  in  the  near  future,  (2)  there  is 
enough  cause-effect  data  to  catch  most  reasons  for  reenlistment/loss,  and 
(3)  the  cause-effect  data  are  supportable  by  accurate  statistics.  However,  to 
gain  experience  on  how  historical  manpower  flows  are  perturbed  by,  for  example, 
a  varying  economy,  he  suggests  using  the  Decisions  Elements  Analysis  (DEA) 
technique  (18).  DEA  is  a  survey  approach  whereby  statistically  significant 
sample  populations  of  personnel  enlisting,  leaving,  and  staying  are  queried  to 
establish  the  main  drivers  affecting  their  respective  decisions.  Modeled 
after  similar  successful  consumer  pro duct -oriented  sales  DEAs,  this  approach 
provides  a  more  accurate  picture  of  how  the  population  will  largely  behave 
during  periods  of  rapid,  sizeable  socio-economic  change.  The  different  splits 
in  the  sample  populations  are  then  used  to  propor tionall y  adjust  the  pools 
within  various  skill  areas  so  that  the  true  manpower  availability  more  closely 
resembles  the  real  world.  Enke  anticipates  that,  with  time,  sufficient 
experience  with  widely  varying  socio-economic  conditions  can  be  gathered  so 
that  personnel  flow  models  can  employ  different  sets  of  probabilities  as  they 
apply  to  the  given  (or  projected)  national  or  international  situation. 

By  comparison,  the  HCM  differs  in  the  use  of  a  starting  manpower  pool 
and  desired  ceiling.  Both  of  these  variables  are  required  data  inputs  to  all 
of  the  previous  models  examined.  The  HCM  calculates  the  desired  manpower 
ceiling  from  Step  2  of  the  methodology  and  does  not  depend  on  a  historical 
starting  pool.  Instead,  the  HCM  uses  an  optimization  routine  initiated  by 
placing  one  hypothetical  individual  (unity)  in  the  first  open-skill  area 
associated  with  a  given  task.  The  value  of  unity  is  multiplied  by  the 
historical  fractional  manpower  flows  and  what  initially  remains  is  a  fraction 
of  an  individual.  The  model  then  proceeds  to  input  more  individuals  in  the 
same  manner,  stepwise,  until  all  the  skill  areas  are  filled  in  accordance  with 
the  manpower  requirement.  This  stepwise  calculation  results  in  the  total 
number  of  people  in  the  pipeline  that  must  be  recruited  and  trained  to 
compensate  for  the  various  personnel  losses.  The  HCM  does  not  appear  to 
address  the  use  of  compensating  factors  to  adjust  its  historical  personnel 
loss  rates  according  to  socio-economic  fluctuations. 

5.  Summary  of  Papers  Related  to  the  HCM  Determination  of  Manpower 

Shortages  (Step  5) 

Step  5  of  the  HCM  draws  on  the  results  of  the  personnel  flow 
analysis  and  ultimately  makes  a  comparison  between  the  pipeline  requirements 
and  the  present  (or  two-year  projection)  of  available  manpower.  The  resultant 
difference  indicates  manpower  shortages  in  the  various  skill  areas.  The  key 
difficulty  with  this  step  is  finding  a  reasonable  estimate  of  available 
manpower.  If  the  manpower  shortage  is  required  for  a  near-term  assessment, 
then  the  existing  manpower  pool  is  a  reliable  number.  However,  if  the 
potential  shortages  must  be  calculated  for  a  system  deployed  five  to  ten  years 
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from  now,  then  the  current  (or  near-future)  availability  number  may  be 
inadequate.  Several  papers  address  the  long-term  manpower  projection 
problem.  All  of  the  studies  (Lindsay's  future  Army  enlistment  regression 
(24),  Barnes'  soldier  90  regression  (25),  and  Grissmer's  year  2000  manpower 
supply  scenario  (26))  were  service-specific.  Both  the  Lindsay  and  Barnes 
papers  were  standard  linear  regression  models  that  project  outyear  manpower 
requirements.  Grissmer  employed  a  non-linear  regression  model  that  further 
adjusted  its  manpower  trends  up  or  down  through  the  use  of  elasticities  (i.e., 
weighting  factors  such  as  declining  economy,  unemployment,  pay  changes,  etc.), 
which  are  projected  to  take  effect  at  various  time  intervals  and  therefore 
distort  the  curve  up  or  down  as  they  occur.  The  major  drawback  of  this 
techniques  is  that  adjustments  are  not  made  to  account  for  the  lower 
confidence  associated  with  outyear  projections. 


6.  Sumnary  of  Papers  Related  to  the  HCM  Tradeoff  Analysis  (Step  6) 

The  last  step  in  the  HCM  is  reflective  in  that  various  "what  if" 
scenarios  are  developed  in  response  to  excessive  costs  or  manpower  and 
training  shortages  revealed  by  the  results  of  Step  5.  Many  of  the  "what  if" 
scenarios  revolve  around  cost  because  it  is  a  conmon  denominator  for  many 
design  decisions.  In  performing  the  various  tradeoff  analyses  as  a  function 
of  cost,  the  HCM  usually  does  not  extend  beyond  those  elements  directly 
associated  with  MPT  (e.g.,  tradeoffs  related  to  billet  definition  and  costs, 
task  definition,  system  reliability,  task  training,  or  training  devices). 

Other  life-cycle  cost  elements  such  as  spares,  support  and  test  equipment,  and 
technical  documentation  are  not  addressed. 

One  paper  summarized  in  Section  II  was  the  HARDMAN  Life-Cycle  Cost  Model 
(HLCCM)  developed  for  the  Navy  (8).  This  model  was  specifically  designed  for 
performing  a  wide  range  of  system  tradeoffs.  The  complete  program  consists  of 
several  interactive  life-cycle  cost  models  that  allow  the  designer  to  change 
input  values  for  one  or  more  of  the  life-cycle  cost  variables  and  observe  the 
impact  on  the  relative  cost  of  the  other  support  elements.  The  cost  variables 
include  (1)  production,  (2)  wages,  (3)  spares,  (4)  support  and  test  equipment, 
(5)  maintenance,  (6)  technical  documentation,  and  (7)  maintenance  and  operator 
training.  For  example,  if  a  new,  expensive  acquisition  demands  a  large  number 
of  highly  skilled  maintenance  people,  it  may  be  worthwhile  to  examine  the 
impact  of  reducing  the  tasks  and  skill  level  by  investing  more  money  in 
support  and  test  equipment  to  prevent  the  reliability  from  decreasing  and 
subsequent  spares  from  increasing.  Similar  to  other  life-cycle  cost  models, 
the  HLCCM  technique  seems  to  explore  more  adequately  the  range  of  possible 
cost  tradeoffs. 


7.  Summary  of  Papers  Encompassing  the  Complete  HCM 

One  immediate  implication  of  the  preceding  discussion  is  that 
there  are  apparently  very  few  modeling  processes  that  cover  the  complete  MPT 
arena.  Indeed,  the  HCM  seems  to  be  the  one  process  that  achieves  this  level 
of  coverage.  However,  another  MPT  procedure  examined,  which  appeared  to  be 


30 


designed  to  the  same  level  of  detail,  was  the  Air  Force  "Acquisition  of 
Supportable  Systems  Evaluation  Acquisi tion  Technology  (ASSET)"  methodology 
(27).  One  of  the  reasons  the  Air  Force  procedure  demonstrates  reasonable 
equivalence  to  the  overall  HCM  is  because  ASSET  was  originally  designed  by  the 
same  contractor.  In  its  present  form,  ASSET  represents  the  combined  past  and 
present  design  efforts  of  the  HCM  design  team,  Westinghouse  Corporation,  and 
the  Air  Force  (27). 

In  contrast  to  the  HCMs  six  major  steps,  ASSET  has  eight: 

(1)  Define  the  specific  application  and  identify  major  data  sources 
and  planning  documents. 

(2)  Construct  the  consolidated  data  base  (similar  to  the  HCM  CDS). 

(3)  Define  all  operator  and  maintenance  tasks,  tools,  and  test  or 
support  equipment. 

(4)  Develop  maintenance  action  networks. 

(5)  Establish  other  logistic  support  resources  based  on  the  task 
analysis  and  maintenance  network. 

(6)  Compare  the  new  system  alternative  designs  against  a  baseline 
system  to  identify  different  maintenance  and  resource  demand 
scenarios . 

(7)  Perform  a  life-cycle  cost  assessment  using  all  of  the  above 
logistics  support  inputs. 

(8)  Develop  a  design  option  decision  tree  that  indicates  various 
tradeoff  situations  critical  to  logistics  and  human  resources. 

Although  the  Air  Force  technique  seems  to  use  more  detailed  modeling 
routines,  it  admits  to  having  some  data  availability,  data  manipulation,  and 
subroutine  integration  problems  (e.g.,  problems  with  obtaining  data  for 
maintenance  network  probabilities  or  the  mismatch  between  the  ASSET  personnel 
availability  model  and  the  equivalent  information  generated  by  the  Air  Force 
Manpower  and  Personnel  Center)  (27). 

In  summary,  it  seems  that  the  HCM  does  not  strongly  diverge  from  either 
standard  DOD  MPT  doctrines  or  other  accepted  MPT  modeling  practices.  Although 
distinctions  were  made  throughout  the  preceding  discussion,  the  following 
section  sunmarizes  the  major  positive  and  negative  differences  more  concisely. 

E.  COMPARISON  OF  DOD  AND  OTHER  MPT  GUIDELINES  AGAINST  HCM 

The  major  differences  between  the  HCM  and  the  other  various  MPT  modeling 
techniques  were  pointed  out  in  the  preceding  paragraphs.  In  some  cases  the 
distinctions  of  other  modeling  practices  represent  improvements  because  they 
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address  real-world  considerations  that  could  affect  the  projection  accuracy. 

In  other  situations,  the  HCM  strengths  provide  a  stronger  theoretical  founda¬ 
tion.  Table  5-1  summarizes  the  areas  in  which  the  strengths  and  weaknesses 
were  apparent. 

Table  5-1  clearly  indicates  that  overall  the  HCM  is  fairly  similar  to 
other  MPT  models  in  its  logic,  use  of  associated  MPT  parameters,  and  input 
data.  The  major  differences  considered  as  weaknesses  originated  from  (1)  using 
a  rectangular  manpower  function,  (2)  not  making  allowances  in  the  manpower 
flows  for  the  possible  impacts  of  major  socio-economic  fluctuations,  (3)  not 
considering  long-range  manpower  trends  on  personnel  availability,  and  (A)  not 
considering  all  operations  and  support  variables  in  the  tradeoff  and  life- 
cycle  cost  assessments.  Item  (1)  above  is  important  because  of  the  potential 
life-cycle  end  point  problems  cited  earlier  during  the  discussion  of  Putnam's 
approach.  Enke's  point  reiterated  in  item  (2)  cannot  be  overlooked  because 
some  socio-economic  impacts,  such  as  a  rapid  defense  drop-off  with  change  in 
administration,  are  not  minor  perturbations.  Perturbations  such  as  this  can 
affect  the  supply  of  manpower  for  several  years.  Item  (3)  is  an  important 
consideration  because  the  normal  seven-  to  ten-year  acquisition  period  far 
exceeds  the  two-year  manpower  planning  projection  used  in  the  Army  and  HCM. 
Admittedly,  the  Grissmer,  Lindsay,  and  Barnes  papers  referred  to  in  Table  5-1 
do  not  include  a  confidence  factor  in  their  manpower  projections.  However, 
the  use  of  confidence  intervals  is  standard  practice  and  could  easily  be 
incorporated.  Recommendations  such  as  this  are  discussed  in  greater  detail 
later  in  this  report.  The  last  area,  item  (4),  was  considered  a  weakness 
because  all  life-cycle  elements  must  be  considered  to  completely  assess  costs 
and  design  tradeoffs. 

The  HCM  clearly  exceeded  some  of  the  other  studies  in  the  areas  related 
to  building  the  data  base  and  describing  tasks  and  the  personnel  pipeline 
calculations.  The  data  base  and  task  area  descriptions  were  comparatively 
better  than  the  Fong  or  Gordon  papers  because  these  studies  were  not 
projection  models.  The  advantage  of  the  HOI  over  the  series  of  papers  related 
to  the  pipeline  projection  was  the  optimization  routine.  By  not  requiring 
either  manpower  ceiling  or  starting  pool  data  inputs,  the  model  reduced  the 
chance  of  contaminating  the  outputs  with  incorrect  or  obsolete  data. 

Having  identified  same  of  the  strengths  and  weaknesses  of  the  HCM  via 
the  literature  search,  the  next  step  was  to  investigate  some  actual  applica¬ 
tions  of  the  methodology,  drawing  on  the  familiarization  provided  by  the 
preceding  comparative  examination.  Inferences  about  the  overall  face  validity 
of  the  HOI  were  withheld  until  the  audits  were  completed,  and  any  obvious 
parallels  between  the  literature  search  findings  and  applications  analysis 
were  identified.  The  final  results  are  provided  in  the  following  paragraphs. 


F.  USE  OF  AUDITS  TO  EXAMINE  FACE  VALIDITY 

The  audit  phase  of  the  face-validity  analysis  was  designed  to  follow  the 
literature  searches  so  that  the  inquiry  might  be  better  organized  (having 
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obtained  scxae  foreknowledge  of  the  HOI  process  and  its  shortcomings).  As 
stated  earlier  in  this  section,  the  three  applications  selected  for  the  audits 
were  SINCGARS,  RPV,  and  DSWS.  These  three  systems  were  chosen  primarily 
because  they  represented  the  most  recent  applications  of  the  HOI.  These  cases 
were  also  selected  because,  being  at  different  stages  in  the  acquisition 
process,  they  also  typified  different  levels  of  study  detail  that  is  useful 
for  examining  potential  data  availability  or  credibility  issues.  Finally,  the 
RPV  analysis  was  executed  by  a  different  contractor  team  than  the  one 
responsible  for  SINCGARS  and  DSWS. 

As  an  introduction,  SINCGARS  is  a  sophisticated  inter-communication 
radio  system  consisting  of  a  manpack  subsystem  (i.e.,  radio  backpack)  and  six 
other  radio  configurations  that  can  be  mounted  on  different  types  of  vehicles, 
i.e.,  trucks,  tanks,  etc.  (28).  The  RPV,  a  small,  drone-type  aircraft  that  is 
remotely  piloted  from  a  mobile  command  center,  provides  surveillance  informa¬ 
tion  on  enemy  target  locations  (29).  Last,  the  DSWS  system  is  a  self-propelled 
Howitzer  linked  with  three  other  respective  ammunition  resupply,  operations 
center,  and  maintenance  vehicles  (30). 

Each  design  is  fairly  complex  because  several  major  components  make  up 
the  complete  system.  Given  the  time  and  budget  constraints  for  the  total 
evaluation,  it  was  decided  that  only  select  components  from  each  system  could 
be  examined  in  any  depth.  The  three  following  selection  criteria  were 
developed  in  an  effort  to  choose  key  components  that  could  surface  validity 
problems: 

(1)  The  component  should  be  a  major  manpower  driver  in  terms  of 
quantity  and  skills. 

(2)  The  component  should  require  maximum  exercise  of  the  mechanics  and 
judgmental  portions  of  the  methodology  to  explore  validity  issues 
such  as  error  sources,  data  availability,  logic,  and  credibility. 

(3)  Where  possible,  the  component  should  demonstrate  the  least 
quantity  and  quality  of  source  data,  therefore  requiring  an 
exercise  of  the  most  judgmental  portions  of  the  methodology. 

These  criteria  would  provide  insight  into  the  most  likely  areas 
where  error  sources,  credibility,  logic,  or  ambiguity  problems 
could  arise. 

Although  it  was  difficult  to  find  components  that  simultaneously  met  all 
criteria,  generally  the  four  to  six  canponents  selected  for  each  system 
satisfied  at  least  two  of  the  guidelines.  Once  the  studies  were  reviewed  and 
the  components  analyzed,  a  detailed  list  was  assembled  that  addressed  all  the 
areas  where  various  face-validity  issues  could  arise.  The  next  step  was  to 
interface  with  the  HCM  designers  to  trace  and  understand  exactly  what  was  done 
to  confirm  the  problem  areas.  This  was  accomplished  by  converting  the  problem 
list  into  a  questionnaire,  visiting  the  contractor's  facility,  and  replicating 
in  detail  the  analysis  performed  (31).  The  complete  questionnaire  is  shown  in 
Appendix  C. 
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Two  caveats  pertaining  to  the  audits  should  be  stated  at  the  onset. 
First,  only  the  RPV  and  DSWS  audits  were  actually  conducted  at  the  con¬ 
tractor's  facility.  The  SINCGARS  audit  was  performed  off-site  under  the  same 
guidelines  as  a  test  of  the  internal  reliability  of  the  methodology  (32).  The 
test  of  internal  reliability  requires  that  an  independent  group  of  trained 
individuals,  given  the  same  data  inputs,  attempt  to  replicate  the  results  of  a 
previous  application  of  the  methodology.  Although  the  details  of  the  internal 
validity  test  are  provided  later  in  Section  VI,  it  suffices  to  state  here  that 
the  face-validity  issues  surfaced  during  the  on-site  RPV  and  DSWS  reviews  also 
appeared  in  the  separate  SINCGARS  replication. 

The  second  caveat  applies  to  the  presentation  and  interpretation  of  the 
audit  results.  Appendix  C  contains  approximately  eighty  questions  related  to 
both  the  three  specific  systems  and  the  methodology  as  a  whole.  It  should  be 
noted  that  of  the  eighty  issues  examined,  which  covered  the  complete 
methodology,  roughly  83%  (or  68  of  the  potential  issues)  were  adequately 
addressed  and  resolved  during  the  audits.  A  summary  of  the  remaining  problem 
areas  follows. 


1.  Results  of  the  SINCGARS  Audit 

The  major  findings  of  the  SINCGARS  audit  revolved  around  the 
clarity  of  the  methodology,  usability  of  the  technique,  and  data  management. 
The  clarity  issues  primarily  applied  to  the  early  Step  1  and  2  analysis  done 
in  support  of  the  ultimate  manpower  and  personnel  projections.  For  example, 
functional  requirements  had  to  be  developed  as  a  subset  of  selecting 
predecessor  components  to  both  construct  a  reference  system  and  load  the 
historical  data  bank.  The  participants  in  the  replication  found  that  the 
detail  to  which  the  requirements  could  be  defined  had  a  direct  relationship  to 
the  number  of  technically  different  components  selected  to  build  a  reference 
system  and,  therefore,  could  affect  the  number  of  people  needed  to  support  the 
system.  There  were  no  clear  guidelines  provided  for  determining  the  depth  to 
which  the  requirements  should  be  developed. 

Another  area  of  contention  developed  over  the  construction  of  the 
reference  system.  The  reference  system  is  constructed  using  predecessor 
components  that  meet  the  functional  requirements  and,  in  turn,  are  similar  to 
the  proposed  system(s)  one  desires  to  model.  The  study  group  discovered 
through  the  SINCGARS  example  the  possibility  that,  if  two  or  more  contractor- 
proposed  designs  differed  substantially,  it  could  be  very  difficult  to  con¬ 
struct  one  reference  system  similar  to  both  designs.  This  dilemma  could 
result  in  a  weaker  manpower  projection  for  one  case  than  the  other.  Although 
the  obvious  solution  was  to  develop  two  reference  systems,  the  larger  problem 
was  the  matter  of  defining  how  much  two  proposed  designs  could  diverge  before 
one  reference  system  was  insufficient  to  resemble  the  performance  of  both. 

One  area  closely  associated  with  the  manpower  projection  is  the  process 
of  evaluating  design  differences  between  the  predecessor  and  proposed  com¬ 
ponents  and  defining  the  fractional  improvements  in  maintainability  and 
reliability  performance.  The  first  major  difference  occurred  over  the  selec¬ 
tion  of  the  so-called  "perturbation  values,"  which  are  the  ratios  used  to 
adjust  reference  data  to  better  represent  the  performance  of  the  proposed 
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component.  The  study  participants  found  that,  because  there  were  not  estab¬ 
lished  guidelines  for  selecting  perturbation  values,  their  perceptions  of 
certain  design  improvements  varied  widely.  This  variance  could  have  a  major 
affect  on  the  manpower  outcomes. 

The  study  group  suffered  similar  confusion  over  guidelines  associated 
with  the  selection  of  indirect  productivity  and  induced  failure  values. 

Although  distinct  values  were  provided  for  SINCGARS,  the  group  recognized  that 
the  MACRIT,  which  is  the  source  of  these  values,  provides  a  range  of  selec¬ 
tion.  Again,  not  being  given  guidelines  in  the  methodology  on  how  to  select 
either  a  high  or  low  factor  over  a  fairly  wide  range  of  potential  values,  the 
study  participants  felt  that  the  final  manpower  projection  could  be  affected 
considerably . 

The  last,  most  important,  difference  was  associated  with  a  misunder¬ 
standing  of  how  the  perturbation  values  were  actually  used  in  the  manpower 
calculation.  It  was  discovered  that  the  HOI  presently  uses  perturbation 
values  only  to  fill  data  gaps  in  the  proposed  designs.  Occasionally,  HOI 
analysts  find  that  contractors  do  not  provide  performance  data  on  all  the 
components  in  the  proposed  design  (such  as  SINCGARS).  While  constructing  the 
reference  system  out  of  similar  predecessor  components,  the  analysts  will 
select  the  predecessor  component  equivalent  to  the  missing  component  in  the 
proposed  design  and  "perturb"  the  historical  performance  data  in  order  to 
complete  the  data  set  for  the  new  design.  However,  in  the  final  manpower  and 
personnel  projections  the  total  reference  system  predecessor  data  are  not 
upgraded  (using  perturbation  values)  to  allow  the  complete  new  design  to  be 
modeled  based  on  proven  historical  information.  This  procedure  represented  a 
major  logic  problem  to  the  control  group  because  one  of  the  primary  reasons 
for  modeling  new  system  manpower  requirements  is  to  prevent  contractors  from 
being  overly  optimistic  about  the  impacts  of  new  technology  improvements.  By 
not  upgrading  the  old  technology  to  a  roughly  equivalent  proposed  configuration 
and  establishing  a  revised  historical  data  base  against  which  to  equally  com¬ 
pare  proposed  designs,  the  methodology  cannot  test  the  integrity  of  con¬ 
tractors.  Without  making  an  equivalent  technology-based  comparison,  the  same 
trivial  outcome  will  always  result;  the  new  technology  will  appear  to  perform 
better  than  old  technology  (i.e.,  the  new  technology  will  require  either  fewer 
or  lower-quality  skilled  people). 

One  area  of  contention  that  arose  over  the  personnel  assessment  portion 
of  Step  2  in  the  methodology  was  the  matching  process  between  tasks  and 
skills.  The  participants  found  that  the  cookbook  approach  outlined  in  the 
methodology  was  not  completely  accurate.  This  definitive  approach  is  normally 
followed  up  with  the  analysts  actually  contacting  various  Army  training 
schools  to  confirm,  or  update,  the  task  and  skill  matchups.  Because  this 
caveat  was  not  passed  on  to  the  participants,  there  was  general  disagreement 
between  the  skills  selected  by  the  original  methodology  designers  and  the 
skills  selected  by  the  control  group. 

The  last  two  areas  pertaining  to  the  usability  and  data  issues  were  more 
related  to  the  methodology  as  a  whole.  The  analysts  discovered  that,  although 
they  jointly  had  backgrounds  in  mathematics,  engineering,  and  personnel,  they 
still  lacked  the  range  of  training  and  experience  necessary  to  completely 
comprehend  the  pitfalls  and  nuances  of  the  methodology.  The  overall  con¬ 
clusion  was  that  the  methodology  could  not  be  properly  exercised  unless  a 


wel 1- seasoned ,  multidisciplinary  team  having  joint  military,  engineering,  data 
management,  and  personnel  backgrounds  was  employed.  The  last  problem  area, 
data  management,  was  not  necessarily  viewed  by  the  study  participants  as  a 
shortcoming  of  the  methodology.  The  observation  was  made  that  the  complex 
nature  of  the  HPT  issues  alone  caused  the  methodology  to  contain  many  places 
vfaere  errors  could  be  introduced  due  to  (1)  the  prodigious  quantity  of  data 
required  and  (2)  the  large  amount  of  data  manipulation  required. 

2.  Results  of  the  RPV  Audit 

The  RPV  audit  surfaced  several  potential  validity  problems 
involving  elements  of  the  early  data-sorting  process,  manpower  and  training 
projections,  as  well  as  the  later  tradeoff  analyses.  The  most  important 
finding  related  to  the  initial  data  base  step  was  the  lack  of  a  clear  decision 
process  for  selecting  and  using  different  sources  of  performance  data  (e.g., 
reliability  and  maintenance  data).  For  example,  it  was  stated  earlier  in  this 
section  that  the  methodology  normally  requires  historical  data  to  be  used  for 
modeling  new  system  performance.  However,  in  the  RPV  case,  most  of  the 
components  had  already  been  prototyped  so  that  actual  reliability  and 
maintenance  test  data  existed.  Nevertheless,  the  decision  was  made  to  use 
only  some  of  the  contractor  test  daca.  The  remaining  maintenance  data  were 
discarded  and  relaced  by  comparable  information  listed  in  the  design  speci¬ 
fications  that  were  part  of  the  original  proposal.  The  reason  cited  for  this 
action  was  that  experience  dictated  the  contractor  data  was  not  reasonable. 
Another  example  of  non-distinct  direction  for  data  selection  pertained  to 
modeling  component  performance  on  the  small  RPV  using  comparable  components  on 
full-sized  aircraft.  The  reason  the  contractor  opted  to  use  this  information 
was  because  the  Army,  having  no  history  of  using  drones,  did  not  have  com¬ 
parable  predecessor  data.  Closely  associated  with  this  decision  was  the  fact 
that  the  original  designers  did  not  differentiate  between  the  different 
performance  and  environmental  envelopes  of  the  aircraft.  These  differences 
could  affect  the  stress  levels  and  resultant  wearout  rates  experienced  by  the 
RPV  components.  The  confusion  over  the  selection  of  data  sources,  although 
somewhat  linked  to  concerns  about  accuracy,  more  specifically  revolved  around 
(1)  identifying  the  various  data  sources  and  ranking  them  as  a  function  of 
quality  and  (2)  determining  when  to  use  one  source  in  preference  to  another. 

In  the  manpower  and  personnel  step  following  Step  1,  the  analysts  found 
the  same  problems  with  selecting  perturbation  and  induced  failure  values  that 
were  discovered  with  SINCGARS.  Similarly,  the  step  of  matching  skills  with 
tasks  lacked  a  clear  audit  trail  as  to  how  the  final  skills  were  selected. 

This  particular  audit  problem  was  derived  from  the  fact  that  the  Army  did  not 
list  skills  for  operating  and  maintaining  drone-type  aircraft.  Therefore,  the 
HCM  analysts  had  to  search  outside  the  normal  cookbook  procedure  to  define  the 
new  skill  areas  and  obtain  associated  wage  and  training  information.  Regarding 
the  final  manpower  projection  made  in  Step  2  of  the  methodology,  the  step 
function  deployment  assumption  discussed  earlier  in  the  literature  search  was 
confirmed.  As  indicated  before,  this  assumption  is  considered  a  problem 
because  of  the  potential  personnel  allocation  conflicts  imposed  by  other 
acquisitions  under  concurrent  development. 


Step  3  of  the  methodology,  the  training  resource  assessment,  surfaced 
some  interesting  problems  with  respect  to  RPV.  The  analysts  discovered  that, 
although  the  methodology  provides  for  a  very  detailed  instructor,  training 
resource  and  cost  asessment,  it  ha6  only  been  marginally  exercised  because  of 
lack  of  data.  Critical  training  considerations  such  as  the  identification  of 
safety  related  tasks  or  task  difficulty  were  not  highlighted.  Additionally, 
the  audit  team  found  that  the  methodology  did  not  differentiate  between  the 
instructor  contact  hours  required  for  critical  versus  non-critical  tasks.  The 
analysts  felt  that  this  was  particularly  important  because  RPV  appeared  to  be 
a  new  technology  area  not  previously  employed  by  the  Army. 

An  in-depth  investigation  of  the  personnel  shortage  projection  in  Step  5 
of  the  RPV  study  was  very  revealing.  The  investigators  found  that,  although 
the  modeling  algorithm  for  calculating  shortages  was  sound,  the  weakness 
pertaining  to  the  use  of  the  short-term  manpower  availability  projection  cited 
in  the  previous  literature  summary  was  verified. 

In  the  final  step  of  the  RPV  study,  tradeoff  analysis  and  conclusions, 
the  analysts  experienced  considerable  confusion  over  the  criteria  for 
(1)  performing  certain  tradeoffs  and  (2)  identifying  the  one  MPT  projection 
that  represented  the  best  estimate  of  the  real-world  performance  of  the 
proposed  system.  For  example,  in  the  training  assessment  several  cases  were 
found  where  the  design  dictated  the  use  of  low  pay  grades  in  certain  task 
areas.  However,  when  the  training  courses  were  structured  based  on  pre¬ 
decessor  systems,  it  was  found  that  rather  high  skills  were  actually  required 
for  the  same  tasks.  Although  the  study  indicated  that  this  mismatch  was  a 
major  conflict,  the  training  and  cost  implications  were  never  pursued  in  the 
tradeoff  analysis.  This  ambiguity  about  the  relative  importance  of  various 
tradeoffs  did  not  give  the  analysts  confidence  that  (1)  all  possible  tradeoffs 
had  been  considered;  (2)  important  tradeoffs,  such  as  the  above  example,  had 
not  been  overlooked;  and  (3)  the  tradeoffs  finally  pursued  represented  the 
pivotal  issues.  The  largest  source  of  ambiguity  in  the  RPV  study  revolved 
around  the  presentation  of  the  final  MPT  and  cost  projections.  The  present 
summary  format  for  the  projections  displays  both  the  reference  and  proposed 
results  together.  However,  where  wide  variations  between  the  projections 
occurred  (such  as  SINCGARS)  or  when  several  proposed  systems  were  presented 
(such  as  RPV),  it  was  not  clear  which  set  of  results  represented  the  most 
accurate  approximation  and  MPT  planning  guide.  The  audit  team  found  that  the 
source  of  the  confusion  originated  from  not  having  an  indication  of  the 
relative  credibility  of  the  data  inputs  and  outcomes. 

One  final  interesting  aspect  noted  by  the  analysts  applied  specifically 
to  RPV  but  not  to  SINCGARS.  The  observation  was  made  that  because  the  RPV 
study  was  completed  roughly  one  year  before  this  evaluation,  the  members  of 
the  original  HCM  study  team  did  experience  memory  decay  with  respect  to  their 
decisions  and  judgments  about  the  use  of  certain  data  sources  and  the  ultimate 
selection  of  one  of  the  proposed  system  designs  as  the  best  planning  figure. 
This  last  finding  was  important  because  it  demonstrated  that,  even  with  a 
seasoned  corporate  bank  of  knowledge,  the  judgment  areas  in  the  methodology 
must  be  better  structured  to  allow  a  firmer  audit  trail. 
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3. 


Results  of  the  DSWS  Audit 


The  findings  of  the  DSWS  audit  were  quite  similar  to  both  SINCGARS 
and  RPV.  The  main  reasons  for  the  similarities  were  (1)  the  DSWS  reference 
system  was  compared  against  several,  widely  varying  proposed  designs;  (2)  a 
significant  amount  of  data  sorting  and  selection  was  required;  and  (3)  the 
original  DSWS  analysis  was  also  completed  well  before  the  methodology  validity 
evaluation.  The  chief  problems  encountered  by  the  investigators  related  to 
item  (1)  above  were  those  associated  with  defining  the  proper  level  of  detail 
for  the  functional  requirements  and  building  one  reference  system  to  model 
four  widely  varying  contractor  concepts.  Like  the  SINCGARS  audit,  it  was  felt 
that  the  lack  of  sufficient  detail  on  functional  requirements  and  singular 
reference  system  constraints  could  cause  major  variance  and  errors  in  the 
operational  and  maintenance  manpower  support  projections.  Item  (2)  above  was 
extremely  important  because  of  the  confusion  generated  in  trying  to  sort  and 
categorize  generic  component  repair  times  by  specific  task,  establish  the 
credibility  of  the  various  data  sources,  and  understand  which  data  sources  had 
priority  over  others.  The  DSWS  data  question  was  analogous  to  the  earlier 
defined  RPV  data  problem.  Item  (3)  addresses  the  audit  trail  decay  dilemma 
similarly  revealed  with  the  RPV  study. 

As  was  the  case  with  both  the  SINCGARS  an  RPV  investigations,  the 
analysts  did  not  clearly  see  the  selection  criteria  for  the  DSWS  perturbation 
values,  indirect  productivity  factors,  and  final  choice  of  skills.  Finally, 
the  investigators  were  confused  by  the  way  the  data  were  presented  for  the 
final  design  option  selected  as  the  MPT  planning  guide.  Similar  to  RPV  and 
SINCGARS,  the  study  team  felt  that  the  best  option  should  have  been 
respectively  chosen  as  a  function  of  (1)  the  relative  quality  of  the  various 
data  inputs  and  (2)  a  comparison  against  a  reference  system  that  had  been 
perturbed  so  that  the  historical  and  proposed  designs  were  compared  on  an 
equal  technological  basis. 


G.  VALIDITY  IMPLICATIONS 

The  preceding  discussion  highlighted  many  strengths  and  weaknesses  of 
the  HQ1.  The  overriding  strengths  were: 

(1)  The  methodology  conformed  well  with  DM)  and  Army  doctrines. 

(2)  The  technique  basically  resembled  other  known,  accepted  MPT 
modeling  techniques  in  its  logic,  use  of  MPT  variables,  and  input 
data. 

(3)  The  methodology  was  reasonably  well  supported  by  its  data  sources. 

(4)  The  HCM  surpassed  most  other  models  in  its  completeness  and 
integration  of  various  MPT  modeling  techniques. 

(5)  The  methodology  provided  a  better  modeling  foundation  in  some 
areas  than  other  simulations  (e.g.,  manpower  pipeline  projections). 

(6)  The  HCM  stood  up  reasonably  well  under  an  extensive  audit  (i.e., 

83  percent  of  the  test  issues  were  adequately  answered). 
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However,  Che  audits  confirmed  some  problem  areas  outlined  in  the 
literature  search.  During  the  process  of  replicating  the  three  case  studies 
the  remaining  15  percent  of  the  test  issues  revealed  that  (1)  the  HCM 
conformed  with  the  MACRIT  in  its  use  of  a  rectangular  manpower  curve  with  no 
consideration  of  the  Norden-Raleigh  reduced  end  point  demands;  (2)  the  HQ4  did 
not  address  Enke' s  potential  impacts  of  major  socio-economic  fluctuations  on 
personnel  flows  in  and  out  of  various  skills,  and  the  Army  as  a  whole;  (3)  the 
technique  did  not  project  manpower  availability  and  shortages  at  the  actual 
projected  deployment  time  of  the  new  system;  (4)  the  HOI  incorporated,  but  did 
not  exercise,  a  technique  similar  to  Winston's  subjective  critical  task 
assessment;  and  (5)  che  HCM  MPT  system  examined  only  a  small  number  of  life- 
cycle  cost  tradeoffs  (e.g.,  wages,  reliability  changes,  task  redefinition, 
training  programs  and  devices).  Additionally,  in  the  data  base  construction 
phase,  early  in  Step  1  of  the  methodology,  there  was  a  wide  variance  in  (1) 
the  types  and  sources  of  data  selected,  (2)  the  reliability  and  maturity  of 
the  data,  and  (3)  the  methods  used  in  screening  and  manipulating  the  data  to 
conform  to  the  methodology  or  to  fill  data  gaps.  This  wide  variance  caused  a 
major  problem  in  establishing  the  credibility  of  both  the  analysis  and  final 
projections.  It  was  also  discovered  in  Step  1  that,  by  not  reaching  the 
proper  level  of  indenture  for  the  functional  requirements,  key  tasks  and 
subsequent  manpower  support  could  be  left  out.  Another  major  finding 
associated  with  the  reference  system  portion  of  Step  1  was  particularly 
apparent  in  the  SINCGARS  and  DSWS  audits.  It  was  discovered  that  where 
several  widely  varying  new  designs  were  being  considered,  the  use  of  a  single¬ 
reference  system  could  be  an  inadequate  performance  comparator  for  some  of  the 
proposed  concepts.  Also  linked  to  the  reference  system  procedure  was  the 
problem  of  not  adjusting  the  predecessor  historical  data  as  a  function  of  the 
proposed  design  improvements.  This  was  a  pivotal  discovery  because  the 
contractor-projected  manpower  and  costs  could  not  be  assessed  on  an  equal 
basis  with  vrtiat  would  be  considered  a  reasonably  equivalent  extension  of 
existing  technology. 

The  manpower  and  personnel  projection  phase  of  the  audits  uniformly 
revealed  that  a  better  logic  structure  was  required  for  selecting  perturbation 
values,  indirect  productivity  factors,  and  induced  failure  rates.  It  was 
found  that  arbitrary  selection  of  these  different  values  would  affect  the 
ultimate  manpower  projection,  particularly  where  small  variations  in  manpower 
support  (such  as  at  the  field  level)  could  cause  critical  shortages.  In  the 
personnel  selection  step  it  was  found  that  HCM' s  actual  skill  matching  process 
extended  beyond  the  scope  of  the  stated  methodology.  The  resultant  skill 
mismatch  and  cost  impacts  were  felt  to  be  potentially  sizeable. 

In  the  tradeoff  and  conclusion  portion  of  the  methodology,  Step  6,  the 
audits  revealed  a  definite  need  for  (1)  a  clear  structure  for  selecting 
tradeoffs  and  (2)  a  subjective  weighting  mechanism  of  the  various  data  and 
judgmental  elements  of  the  methodology  to  establish  the  credibility  of  both 
the  final  design  selected  and  the  accuracy  of  the  analysis  as  a  whole. 

Overall,  the  findings  of  the  audits  suggested  that  the  structure  of  the 
judgmental  portions  of  the  methodology  still  needed  to  be  improved  to  offset 
the  observed  audit  trail  decay.  Furthermore,  the  sophistication  of  both  the 
MPT  problem  and  the  HCM  design  did  not  make  the  process  amenable  to  use  by  lay 
people.  Although  the  preceding  discussion  indicates  the  various  sources  of 
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potential  errors,  logic  problems,  ambiguities,  or  credibility  issues,  Table  5-2 
more  concisely  illustrates  the  link  between  these  findings  and  complete  array 
of  respective  face-validity  elements  defined  earlier  in  this  section. 

The  face-validity  analysis  provided  an  indication  of  the  strengths  and 
weaknesses  of  the  HCM,  as  well  as  a  first-cut,  general  examination  of  the 
integrity  of  the  technique.  The  next  step  was  to  investigate  and  tailor 
validity  tests  to  examine  more  specifically  the  accuracy  and  reliability  of 
both  the  overall  methodology  and  its  projections.  The  validity  tests  selected 
for  the  reliability  portion  of  the  study,  selected  to  conform  with  the  project 
time  and  budget  constraints  discussed  in  Section  III-E,  are  discussed  in 
detail  in  Section  VI  along  with  the  results  of  the  complete  reliability 
analysis . 


Table  5-2.  Summary  of  Audit  Results  in  Terms  of  Face-Val idity  Elements 
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shortages  based  on  actual 
system  deployment  data 


SECTION  VI 


RELIABILITY  ANALYSIS 


A.  OVERVIEW 

As  explained  in  Section  III,  the  reliability,  or  accuracy,  of  the  HCM 
was  explored  via  three  avenues.  The  reliability  of  the  HARDMAN  methodology 
was  assessed  by  examining  internal  reliability,  to  a  limited  extent  an 
examination  of  the  experience  of  individuals  who  have  already  applied  the  HCM 
(or  other  similar  methodologies)  to  real-life  applications,  and  event  series 
validity. 

For  the  internal  reliability  test,  it  was  decided  to  use  one  of  the  Army 
test  cases  (i.e.,  SINCGARS).  A  controlled  experiment  was  designed  to 
establish  whether  a  group  of  individuals,  trained  to  use  the  methodology, 
could  both  replicate  the  original  answers  and  also  demonstrate  a  low  degree  of 
variance  in  their  answers.  A  group  of  seven  individuals  (four  civilians  and 
three  career  Army  soldiers)  of  varying  backgrounds,  were  selected  for  the 
experiment.  Given  the  small  sample  size  (primarily  due  to  restrictions  in 
available  people,  time,  and  budget)  it  was  decided  to  dispense  with  detailed 
statistical  comparisons  and  simply  establish  general  conclusions  about  the 
repeatability  and  consistency  of  the  results.  The  replication  was  also  used 
to  develop  a  list  of  critical  factors  that  have  major  impacts  on  MPT 
projections.  For  purposes  of  consolidating  the  internal  reliability  findings 
and  to  find  ways  to  improve  future  versions  of  the  HARDMAN  methodology,  each 
participant  went  through  a  formal  debriefing.  Furthermore,  a  discrepancies 
resolution  conference,  attended  by  the  study  group,  ARI,  and  HCM  designers, 
was  held  following  the  completion  of  the  replication. 

Another  important  step  in  the  reliability  examination  was  a  qualitative 
accuracy  check  of  past  Air  Force  and  Navy  experience  with  the  same  (or 
reasonably  close)  methodology.  This  was  done  by  personally  contacting 
pertinent  project  personnel. 

The  last  step  in  the  reliability  examination  was  a  manpower  sensitivity 
analysis.  Sensitivity  analysis  is  a  useful  tool  for  testing  how  well  a 
technique  like  the  HCM  must  conform  with  the  real  world.  This  type  of  test  is 
often  referred  to  as  an  "event  series  validity  test"  because  it  ultimately 
provides  an  acceptable  accuracy  or  reliability  envelope  for  the  outcomes  of  a 
modeling  process.  If  the  envelope  is  rather  large,  then  a  quick-and-dirty 
approximation  is  acceptable.  As  the  envelope  shrinks,  greater  emphasis  must 
be  placed  on  improving  the  quality  and  accuracy  of  each  data  input  and 
algorithm.  In  this  evaluation,  the  sensitivity  (or  event  series)  analysis  was 
done  last  to  assist  in  formulating  and  prioritizing  the  closing  recommenda¬ 
tions  for  improving  potential  problem  areas  in  the  methodology.  To  obtain  an 
indication  of  the  required  accuracy  of  the  HCM,  a  simple  top-down  cost 
sensitivity  analysis  was  designed  to  establish  the  allowable  variance  in  the 
manpower  projections  which  would  still  provide  accurate  MPT  estimates.  The 
estimates  of  the  allowable  variance  in  the  manpower  projections  were  performed 
for  one  sample  acquisition  using  an  attual  Army  system.  Each  element  of  the 
reliability  analysis  is  described  in  greater  detail  in  the  next  subsection. 
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B 


APPROACH 


1.  Internal  Reliability 

The  replication  effort  for  evaluating  internal  reliability 
involved  the  application  of  the  first  two  steps  of  the  HARDMAN  methodology  to 
the  Army's  Single  Channel  Ground  Airborne  Radio  System  (SINCGARS).  These  two 
steps  were: 

(1)  Establishing  a  consolidated  data  base. 

(2)  Determining  manpower  requirements. 

The  remaining  four  steps  were  not  included  for  several  reasons.  First, 
preliminary  review  of  the  HCM  process  suggested  that  there  were  a  substantial 
number  of  judgments,  having  considerable  influence  on  the  manpower  estimates, 
in  the  first  two  steps.  Second,  at  the  time  the  replication  was  performed, 

HCM  designers  had  only  completed  their  analysis  on  the  first  two  sections  of 
the  methodology.  Third,  other  duty  commitments  mandated  that  the  experiment 
be  simplified  so  that  the  Army  participants  could  finish  the  replication 
within  a  maximum  time  frame  of  three  weeks.  The  SINCGARS  components  included 
in  the  replication  were  as  follows: 

(1)  Receiver/Transmitter  Unit  (R/T). 

(2)  Electronic  Counter-Countermeasures  Unit  (ECCM). 

(3)  ECCM  fill  device. 

(4)  Securable  Remote  Control  Unit  (SRCU). 

These  components  were  confirmed  as  offering  the  greatest  number  of  judgment 
exercises . 

A  group  of  seven  individuals  with  varying  technical  backgrounds  (i.e., 
math,  engineering,  psychology)  and  experience  (JPL,  Soldier  Support  Center, 

Army  Research  Institute)  were  selected  for  the  testing.  This  group  attended 
the  HCM  training  course,  which  was  held  at  the  HCM  designer's  facility  in 
Boston,  Massachusetts,  during  February  1983.  The  goal  of  the  course  was  to 
familiarize  the  participants  with  the  key  analytical  and  judgment  areas  in  the 
first  two  steps  of  HARDMAN.  During  the  training  sessions,  the  methodology  was 
applied  to  an  example  subsystem  of  the  Army’s  Multiple  Launch  Rocket  System. 

For  the  SINCGARS  replication  effort  the  two  steps  were  divided  into 
eight  different  data  packages.  Each  package  required  the  determination  of  key 
judgment  and  calculational  outputs  and  was  preceded  by  (1)  a  problem  statement 
related  to  the  judgments  that  were  required,  (2)  a  list  of  the  enclosed  data 
items  to  facilitate  making  the  judgments  and  necessary  calculations,  and  (3)  a 
brief  statement  about  how  to  use  the  data.  The  problem  statements  and  descrip¬ 
tions  of  the  data  packages  are  given  in  Table  6-1. 

In  general,  the  participants  were  to  proceed  independently  and 
sequentially  through  the  eight  data  packages  until  the  four  components  of 
SINCGARS  identified  for  the  replication  effort  were  evaluated.  Three  weeks 


Table  6-1.  Data  Package  Problem  Statements  and  Descriptions 


Problem  Statement 


Date  Package  Description 


Develop  system  functional 
requirements 

Construct  reference  system 


Define  tasks 


Reliability  and  maintenance 
determinations  for  inherent 
versus  induced  component 
failures  and  assignment  of 
task  echelons 

Determine  design  differences 
(design  difference  analysis, 
impact  of  design  differences, 
derivation  and  application  of 
perturbation  values  (PV)  to 
appropriate  tasks) 

Select  and  assign  MOS 


System/component 
usage  rates 

Determine  workload  and 
manpower  requirements 


Descriptive  data  for  proposed  system 
and  one  worksheet  for  each  subcomponent 

Candidate  list  of  predecssor  systems 
and  description  of  contractor  proposed 
systems 

List  showing  range  of  generic  classes  of 
tasks  and  contractor  task  definitions 
for  proposed  systems 

Task  time  data,  component  failure  data, 
induced  failure  data,  equipment  break¬ 
down  structures,  predecessor  task  echelon 
assignments  -  one  worksheet  for  each 
component 

Data  extracted  from  1,  2,  and  4,  above, 
summaries  of  contractor  correspondence 
with  component  contractors  and  one 
worksheet  for  each  component 


List  of  candidate  MOS  from  which  to 
choose  appropriate  specialties  for  each 
task 

Select  data  extracted  from  the  operational 
and  organizational  plan 

Data  extracted  from  1-7,  above,  available 
manhours  -  one  worksheet  for  summary  of 
results 


were  allowed  to  complete  the  SINCGARS  replication.  The  study  group  was 
permitted  to  ask  the  HCM  designers  questions  of  clarification  only  by  way  of  a 
mediator.  It  was  originally  planned  to  provide  feedback  to  the  students  at 
selected  audit  points  in  the  analysis  for  the  first  two  SINCGARS  components 
(to  allow  mid-course  corrections)  and  to  provide  no  feedback  for  the  last  two 
components  to  get  a  feel  for  what  the  end-point  variance  might  actually  be, 
given  no  student  coaching.  In  actuality,  because  of  the  huge  quantity  of  data 
involved  and  the  relatively  short  time  frame,  feedback  was  given  to  the 
students  after  each  data  package  completion  for  all  four  components. 
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The  internal  reliability  was  primarily  assessed  by  reviewing  the  study 
group's  answers  to  the  data  packages  and  by  comparing  them  to  the  baseline  HCM 
designer  answers.  Besides  a  reliability  measure,  convergence  of  answers  would 
suggest  that  the  judgment  activities  could  be  done  by  any  individual  with 
appropriate  training  whereas  divergence  would  indicate  inadequacies  with  the 
methodology,  student  background  or  experience,  or  HCM  training  (or  possible 
combinations  of  these  problems).  Another  major  step  in  the  analysis  was  a 
structured  telephone  debriefing  of  each  participant  to  identify  subjectively 
major  problem  areas  in  the  replication  or  methodology. 

The  analysis  of  each  data  package  within  the  reliability  effort  involved 
either  a  subjective  review  of  the  participants'  responses  or  a  numerical 
evaluation  of  the  key  answers  as  compared  to  those  of  the  HCM  design  team.  To 
protect  the  confidential  nature  of  the  replication,  the  participants  were 
referred  to  by  number  only  (from  1  to  6).  After  the  replication  was 
completed,  each  participant  was  debriefed  to  define  individual  problem  areas 
with  the  packages  and  to  get  feedback  for  possible  future  improvements  in  the 
methodology.  The  debriefing  specifically  addressed  relevance  to  present 
assignments,  difficult  portions  of  the  replication,  discrepancies  between  the 
HCM  designer  and  study  group  answers,  and  the  adequacy  of  the  HARDMAN  training 
course  (see  Appendix  D  for  a  sample  debriefing  questionnaire). 

The  last  major  component  of  the  internal  reliability  analysis  was  a 
discrepancies  resolution  conference,  which  was#  held  during  August  1982  at  the 
Soldier  Support  Center  at  Ft.  Benjamin  Harrison,  Indiana.  This  conference  was 
attended  by  the  study  group,  representatives  of  JPL,  the  Army  Research 
Institute,  and  the  HCM  designers.  The  purpose  of  the  meeting  was  to  provide  a 
structured  means  of  resolving  ’ifferences  and  misunderstandings  resulting  from 
the  replication.  This  conference  was  a  two-day  activity  in  which  each  of  the 
eight  data  packages  was  examined  and  discussed  in  detail. 

2.  Qualitative  Accuracy  Check 

The  qualitative  accuracy  check  was  conducted  by  using  participants 
in  existing  military  projects  that  have  used  some  form  of  similar  manpower 
estimating  methodology.  The  projects  covered  included  the  Advanced  Lightweight 
Torpedo  (ALWT),  the  DDGX  destroyer,  the  VTXTX  flight  trainer,  and  the  SUBACTS 
submarine  combat  system.  The  pertinent  project  personnel  were  asked  their 
opinions  on  the  validity  of  the  HARDMAN  methodology  as  compared  to  other 
methodologies,  as  well  as  real-life  data. 

3.  Manpower  Sensitivity  Analysis  (Event-Series  Validity) 

The  objective  of  the  sensitivity  effort  was  to  roughly  approximate 
the  permissiuie  variation  in  the  manning  variable.  To  accomplish  this  objec¬ 
tive,  four  tasks  were  undertaken.  The  first  was  to  develop  a  data  set  of 
actual  costs  for  the  SINCGARS  system.  The  second  was  to  develop  approximate 
manning  data  for  SINCGARS.  The  third  task  was  to  determine  broad,  acceptable 
limits  of  error  on  total  life-cycle  cost  estimates  according  to  authorities 
engaged  in  the  budget  and  acquisition  review  process.  The  final,  and  largest, 
task  was  to  determine  the  extent  of  error  allowable  in  manpower  and  training 
cost  estimates,  given  that  other  cost  elements  cannot  exceed  the  limits  of 
their  allowable  ranges. 
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Within  Task  1,  data  were  collected  reflecting  typical  costs  of  various 
elements  of  life-cycle  cost.  The  major  components  of  total  cost  were 
identified  in  an  attempt  to  determine  approximate  ranges.  It  was  decided  that 
if  possible,  the  more  dif ficult-to-identify  elements  of  life-cycle  cost  would 
be  ignored  if  their  relative  sizes  were  small.  In  Task  2,  actual  training, 
manpower,  and  training  estimates  associated  with  SINCGARS  were  determined.  In 
Task  3,  a  variety  of  participants  in  the  acquisition  process  were  interviewed 
to  determine  what  guidelines  they  use  for  the  range  of  acceptability  of 
errors.  The  groups  contacted  included  OSD  or  CAIG,  who  are  involved  in  the 
review  of  DOD  acquisitions.  The  results  of  these  interviews  were  resolved 
into  perceived  admissible  limits  of  error  on  total  life-cycle  cost  and  each  of 
its  components.  In  Task  4  the  allowable  variation  in  manpower  cost  in  the 
presence  of  other  acceptable  errors  in  the  estimate  was  determined.  This 
study  used  an  already  developed  life-cycle  cost  and  level-of-repair  model  to 
study  the  interrelationships  between  projection  errors  for  several  quantities 
(8).  The  central  idea  was  to  determine  how  large  or  small  the  permissable 
error  in  manning  estimates  could  be  as  a  result  of  these  relationships. 

The  first  step  in  Task  4  was  to  identify  all  the  variables  that 
influenced  manning  and  manning  estimates.  Next,  other  readiness-critical 
quantities  estimated  in  the  model  that  depended  on  the  same  inputs  were 
identified.  There  are  five  major  sources  of  variance  in  manning  require¬ 
ments.  These  are  the  equipment  operating  program,  the  deployment  plan, 
reliability,  maintainability,  and  manpower  characteristics.  Together  they 
determine  actual  manning,  and  estimates  of  the  variables  that  describe  these 
areas  subsequently  impact  the  accuracy  of  manning  estimates. 

The  operating  program  describes  the  number  of  hours  during  which  the 
force  is  expected  to  operate  the  equipment.  This  might  be  twenty  hours  a 
week,  one  or  one  hundred  sixty-eight.  During  the  life  of  the  system,  these 
numbers  will  aggregate  to  a  total  usage  that  will  have  a  fundamental  effect  on 
total  life-cycle  cost.  But  for  any  given  period  within  the  life  cycle,  the 
rate  or  pace  of  operations  will  help  determine  the  demand  rate  for  repairs. 

The  deployment  plan  simply  tells  how  many  units  will  be  purchased  and 
deployed  in  the  field.  By  extrapolation,  the  equipment  operating  hours  are 
determined,  both  over  the  entire  life  cycle  and  per  unit  of  time.  The 
deployment  plan  also  usually  explains  how  usage  rates  are  distributed  among 
operating  units.  This  is  important  because  of  the  effect  of  geographic 
separation  on  various  costs  (i.e.,  pipeline  delay  times,  number  of  maintenance 
facilities,  etc.).  For  example,  a  division  may  generate  the  requirement  for 
only  one  half  a  manyear  of  labor  per  year  to  repair  a  type  of  equipment. 

While  most  labor  costs  would  therefore  be  accounted  for  at  half  cost,  training 
would  not,  because  a  whole  man  must  be  trained  for  each  division. 

The  reliability  of  an  equipment  expresses  the  rate  at  which  failures 
occur  as  a  result  of  usage.  In  company  with  the  operating  program  and 
deployment  plan  which  determine  the  usage  rate,  the  reliability  measure 
(either  mean  time  between  failures  (MTBF)  or  failure  rate)  gives  the  demand 
rate  for  repair  services.  These  three  areas  jointly  determine  how  many 
failures  must  be  repaired  per  unit  of  time. 
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The  maintainability  of  an  equipment  is  expressed  by  the  average  time 
required  to  repair  it.  In  most  systems  this  can  be  divided  into  two  repair 
times.  The  first  is  system  restoration  time  estimated  as  the  mean  time  to 
remove  and  replace  a  faulty  component  (MTRR).  The  second  is  the  mean  time  to 
repair  the  faulty  component  after  the  system  has  been  restored  (MTTR).  The 
system  support  policy  is  usually  developed  by  analyzing  the  relative  costs  of 
undertaking  these  two  repair  actions  in  different  ways  and  at  different 
places.  An  item  is  discarded  if  the  cost  of  repairing  it,  including  all  the 
training,  equipment  and  manpower  consumed,  exceeds  the  cost  of  buying  a 
replacement.  When  the  converse  is  true,  then  the  difference  between  repair  on 
the  spot  versus  repair  at  a  depot  must  be  computed.  At  the  organizational 
level,  the  system  is  generally  repaired  through  fault  isolation  and  replace¬ 
ment  of  the  failed  module.  The  failed  module  may  then  either  be  repaired  or 
discarded. 

The  last  major  determinants  of  manning  are  characteristics  of  the 
manpower  itself.  These  will  include  quality  (skills)  and  a  version  of 
quantity.  The  quality  of  manpower  will  have  an  effect  on  repair  times. 
Generally  speaking,  these  skills  are  set  at  the  lowest  level  capable  of  being 
trained  through  the  service  schools  and  cannot  be  dealt  with  in  the  estimation 
process.  Quantity  is  related  to  the  amount  of  work  available  from  a  single 
individual  in  a  period  of  time.  Typical  times  are  40  to  60  hours  a  week. 
Depending  on  the  circumstances,  these  limits  may  also  be  exceeded. 

It  should  be  noted  that  other  LCC  variables  such  as  special  support 
equipment,  tooling,  etc.,  can  effect  manpower  and,  therefore,  represent 
additional  manning  drivers.  However,  for  the  purposes  of  this  top-level 
sensitivity  study,  variables  such  as  these  were  not  major  manpower 
determinants  and  were  therefore  not  considered. 

SINCGARS  was  selected  as  the  system  to  illustrate  the  error  bound 
interrelationships  because  it  had  been  studied  previously  in  the  HARDMAN 
project  and  because  it  represented  a  small  (therefore,  easy  to  model)  but 
important  electronics  system.  The  cost  model  used  to  perform  the  analysis  is 
an  adaptation  of  the  LCC  model  summarized  earlier  in  Section  V  developed  by 
the  Navy  HARDMAN  Office  (8).  The  bulk  of  the  modifications  made  to  the  LCC 
model  were  aimed  at  converting  the  computer  software  to  a  fast  turnaround 
system  suitable  for  extensive  sensitivity  analysis.  In  addition,  certain 
values  were  added  to  the  printed  output,  along  with  the  ability  to  impose 
predetermined  support  policies  at  the  line  replaceable  unit  (organizational) 
level . 

The  model  is  a  level  of  repair  and  life-cycle  cost  model  that  was 
developed  especially  to  be  sensitive  to  the  relationships  that  drive  manpower 
costs.  It  is  roughly  based  on  the  MIL  STD  1390  group  of  level  of  repair 
models  although  the  errors  implicit  in  those  models  have  been  corrected,  and 
several  innovations  have  been  added  to  the  computational  procedures  (8). 

An  example  baseline  cost  estimate  is  shown  in  Table  6-2.  The  SINCGARS 
life-cycle  cost  estimate  is  $784  million,  excluding  batteries.  The  batteries 
were  left  out  of  the  cost  estimation  process  because  they  consumed  a  great 
deal  of  computational  time  and  added  little  to  the  analysis.  Manning  (in 
raanyears)  is  sunmarized  in  the  lower  right  corner  of  the  table.  Real  manning 
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Table  6-2.  Baseline  Life-cycle  Cost  Sensitivity  Analysis  Run 


Cost  Elements, 

M$ 

System  Rates 

Life-Cycle  Cost 

783846.0 

Number  of  LRA  Types 

17 

System-Unit  Cost 

11.1 

Number  of  LRAs 

17 

INITIAL  COSTS: 

Confidence  Against 

0.999 

Production 

312296.0 

Stockout 

Maintenance  Training 

1104.2 

Operator  Training 

70280.0 

System  MTBF 

472.3 

Mi sc.  Manpower 

0.0 

System  MTTR 

3.0 

Spares 

2512.2 

S+TE 

67048.6 

Technical 

55.2 

Documentation 

TOTAL 

453296.0 

SUPPORT  POLICY  SUMMARIES: 

LRAs  coded  COD 

0 

OPERATION  AND  SUPPORT  COSTS: 

LRAs  coded  MOD 

9 

Maintenance  Wage 

27457.5 

LRAs  coded  local  repair 

4 

Operator  +  Office 

31019.4 

LRAs  coded  discard 

4 

Wage 

Maintenance  Training 

3390.6 

Operator  Training 

215920.0 

Misc.  Manpower 

0.0 

TRAINING  BILLETS  BY  TYPE: 

Spares 

9577.7 

Equipment  Operators 

28112 

Repair 

1962.7 

Org.  and  Int.  Maint. 

420 

Transportation 

17.6 

Depot  Technicians 

6 

S+TE 

41198.5 

Technical 

5.5 

REAL  MANNING  REQUIREMENTS: 

Documentation 

Org.  and  Int.  Maint. 

205.12 

TOTAL 

330550.0 

Depot  Maintenance 

4.27 

OPERATORS 

7028.00 

requirements  indicate  the  total  number  of  manyears  of  labor,  for  all  skills, 
actually  consumed  by  SINCGARS  each  year.  These  manyears  are  not  necessarily 
consumed  in  full  manyear  increments  because  labor  requirements  are  distributed 
over  a  number  of  geographical  units  (28  divisions,  in  this  example).  The 
effect  of  geographical  distribution  is  to  expand  the  real  manning  requirement 
to  a  larger  number  of  billets  required  for  training  purposes.  Because  it  is 
not  possible  to  train  portions  of  people  or  to  give  people  part  of  the 
training  required,  training  billets  are  constrained  to  be  full  integers;  and 
one  must  be  developed  for  every  individual  who  is  required  to  work  on  the 
system,  no  matter  how  little  time  is  spent  with  it. 


The  most  extensive  manpower  requirement  is  for  operators.  This  shows  up 
in  the  consumption  of  real  manning  requirements  as  well  as  in  the  establish¬ 
ment  of  training  billets.  A  glance  at  the  list  of  costs  also  shows  that,  with 
the  exception  of  production  cost,  operator  training  and  billet  costs  constitute 
the  largest  elements  of  life-cycle  cost  for  the  SINCGARS  example.  However, 
while  operator  costs  appear  to  be  important  in  the  estimation  of  total  cost, 
they  are  unimportant  in  this  study  because  they  are  unresponsive  to  changes  in 
most  of  the  input  variables  that  influence  maintenance  manning.  It  is 
understood  that  the  relationship  between  operators  and  maintainers  may  vary 
from  system  to  system. 

The  SINCGARS  manpack  version  was  modeled  as  a  collection  of  17  (18, 
including  the  battery)  line-replaceable  units.  In  fact,  the  SINCGARS  program 
office  basically  views  the  equipment  as  consisting  of  four  modules  (R/T, 
handset,  whip  antenna,  battery)  among  which  the  R/T  breaks  down  into  LRUs. 
Because  the  other  three  modules  have  only  one  piece  and  are  discard  items 
anyway,  the  equipment's  structure  was  simplified  to  accommodate  the  computa¬ 
tional  problem,  treating  modules  as  LRUs  along  with  the  LRUs  that  make  up  the 
R/T. 

The  model  is  driven  by  inputs  that  describe  the  support  and  deployment 
environment,  the  system  design,  and  the  particulars  of  component  design.  The 
data  used  in  this  exercise  were  selected  as  representative  acquisition  values 
and  are  reported  in  Appendix  E,  along  with  the  component  cost  sheets. 

The  model  operates  by  estimating  life-cycle  cost  for  each  component  in 
four  different  support  postures.  Normally,  the  least  costly  of  these  is 
determined,  and  the  output  data  associated  with  it  are  forwarded  to  the  system 
cost  estimate.  In  the  current  example,  support  policies  had  already  been 
determined,  and  these  were  used  to  override  the  cost-minimizing  routine  of  the 
program. 

At  the  same  time  that  costs  are  aggregated  by  the  system  model,  certain 
variables  of  concern  are  aggregated  as  well.  These  include  reliability, 
maintainability ,  and  system-configuration  summary  data,  as  well  as  an 
accounting  of  the  support  postures  selected  for  the  different  LRUs.  While  it 
is  understood  that  skill  tradeoffs  cannot  be  examined  using  aggregate  costs, 
the  use  of  total  costs  for  a  top-level  sensitivity  study  such  as  this  was 
acceptable.  Finally,  the  cost  tableau  is  broken  down  into  investment  of 
initial  cost  and  operating  and  support  costs.  Initial  costs  include  both 
production  and  logistic  investments.  Drawing  on  both  the  Table  6-2  baseline 
costs  and  perceived  acceptable  LCC  error  bounds,  the  above  cost  model  was 
iterated  until  the  relationship  between  manning  and  the  previously  identified 
five  major  LCC  variables  could  be  clearly  plotted. 

C.  SINCGARS  REPLICATION  RESULTS 

1.  Work  Package  1  -  Develop  System  Functional  Requirements 

The  functional  requirements  data  package  consisted  of  three 

subtasks : 

(1)  Identification  of  functions  and  performance  standards  of 
SINCGARS  Manpack  system. 
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(2)  Organization  of  the  functions  and  standards  into  a 
functional  hierarchy. 

(3)  Allocation  of  the  functions  to  equipment  and  people: 

The  HCM  designers'  answer  sheet  was  a  matrix  that  matched  subsystem 
elements  and  performance  standards  to  functions  (see  Table  6-3  for  a  sample 
answer  sheet). 

There  were  no  numerical  answers  to  evaluate  in  this  package.  Part  of 
the  study  group  adjusted  the  answer  format.  Additionally,  differing  semantics 
among  the  participants  complicated  the  analysis.  That  is,  a  "participant" 
could  mean  the  HCM  designers  but  use  different  wording.  Neither  of  these  last 
two  factors  necessarily  made  an  individual's  answers  incorrect.  In  order  to 
evaluate  the  answers,  it  was  decided  to  determine  whether  or  not  the  par¬ 
ticipants  included  the  major  functional  requirements  in  their  answers.  As 
shown  in  Table  6-4,  none  of  the  respondents  had  the  function  "Operate 
SINCGARS."  However,  the  remainder  of  the  answers  indicated  that  the  study 
group  understood  that  this  was  included.  All  respondents  had  "Communicate" 
and  four  (2, 3, 4, 6)  had  "Accommodate  Mobility,"  but  only  two  had  "Command 
Control."  It  should  be  noted  that  an  "X"  indicates  matching  answers  between 
the  HCM  contractor  and  respondents. 

In  terms  of  format,  one  participant  had  three  extra  element  columns 
(adaptor,  shorting  block,  COMSEC),  while  another  individual  never  used  the 
backpack  column.  Two  individuals  itemized  tasks  rather  than  functions.  Three 
individuals  missed  at  least  some  performance  standards.  On  the  other  side  of 
the  spectrum,  five  of  the  study  group  included  extra  items  such  as  "pass 
through  retransmission"  and  "monitor  net,"  which  the  original  HCM  designers 
did  not  have.  In  this  sense,  the  study  group's  answers  were  more  complete. 

In  the  debriefing  it  was  found  that  two  of  the  group  were  confused  as  to 
the  HCM  usage  of  "end  item."  The  study  group  as  a  whole  was  unsure  about  much 
detail  to  give  in  its  answers.  Regardless  of  some  of  the  above  confusion,  the 
major  functional  areas  were  basically  understood  and  covered  by  the 
participants  even  though  they  were  limited  by  an  open-ended  answer  format. 


2.  Work  Package  2  -  Reference  System  Selection 

The  objective  in  this  package  was  to  select  the  reference  system 
based  on  the  following  given  information: 

(1)  Functional  requirements. 

(2)  Reference  system  candidate  equipment. 

(3)  Proposed  system  design  data. 

The  analysis  of  this  package  was  based  on  the  ability  of  the  replicants 
to  specify  the  four  major  system  components:  R/T,  ECCM,  ECCM  Fill,  and  SRCU. 
Given  the  correct  selection  of  the  radio/transraitter  (AN/ARC-114),  the 
remainder  of  the  components  generally  followed.  As  it  turned  out,  all 


Functional  Requirements  Answer  Sheet 


Table  6-4.  Work  Package  1  -  Respondent  vs.  Dynamics  Research 
Corporation  Results 


Major 

Respondents 

Total 

Number 

Functional  Requirements 

DRC 

1  2  3  4  5 

6 

Correct 

Operate  SINCGARS 

X 

Not  specified  but 
inferred 

Communicate 

X 

X  X  X  X  X 

X 

6/6 

Command  Control 

X 

X 

X 

2/6 

Accommodate  Mobility 

X 

XXX 

X 

4/6 

participants  had  doubts  concerning  the  HCM  designers'  answer.  Several  felt 
that  their  selections  were  better  suited  as  the  baseline  system.  In  the 
discrepancy  review  the  HCM  designers  explained  that  the  114  had  the  most 
available  data.  This  was  obvious  from  a  data  source  matrix  that  was  not  given 
to  the  study  group.  This  mismatch  also  suggested  that  the  group  did  not 
understand  the  relationship  between  comparable  component  selection  and  data 
availability  for  that  component. 

This  package  was  among  the  easiest  to  grade  of  all  the  packages  (see 
Table  6-5  for  the  answer  comparison).  Three  individuals  (3,5,6)  had  the 
correct  R/T.  Two  of  these  three  had  the  remaining  major  components  correct  as 
well.  As  in  Package  1,  the  level  of  indenture  was  again  a  concern.  All 
participants  were  missing  at  least  some  detail.  Minor  subsystems  such  as  fill 
cable  and  battery  were  missed  by  everyone  except  Participant  1.  Two  indivi¬ 
duals  had  some  detail  (i.e.,  programmer  gun,  antenna)  that  was  not  included  by 
the  HCM  contractor.  Aside  from  the  doubts  on  the  HCM  designer  selection,  most 
participants  felt  that  the  material  was  too  technical,  while  two  participants 
felt  that  there  was  not  enough  time  allotted  for  this  task. 

In  summary,  without  knowing  the  complete  HCM  contractor  rules  used  for 
the  reference  system  selection,  and  without  all  necessary  input  information, 
it  was  difficult  to  obtain  the  same  answer. 

3.  Work  Package  3  -  Task  Identification 

The  objective  of  this  part  was  to  take  the  previous  reference 
system  design  as  specified  by  the  HCM  designers,  plus  the  data  included  in 
work  package  3,  and  develop  generic  and  reference  task  lists  and  assign  the 
tasks  to  equipment. 


Table  6-5. 

Work  Package  2  -  Respondent  vs 
Corporation  Results 

.  Dynamics  Research 

Major 

System  Components 

DRC  1 

Respondents 

2  3  A  5 

“6 

Number 

Correct 

AN/ARC-1 1A  R/T 

X 

X 

X 

X 

3/6 

SN  A16/APX  76  ECCM 

X 

X 

X 

2/6 

KYK-13/TSEC  Fill 

X 

X 

X 

X 

3/6 

C-2328/GRA-39  SRCU 

X 

X  X 

X  X 

X 

5/6 

The  answers  to  this  package  were  similar  to  those  in  Package  1  in  that 
there  was  a  fair  amount  of  subjective  matching  of  tasks  to  equipment.  This 
again  complicated  the  grading.  Therefore,  it  was  decided  to  check  for  major 
categories  of  equipment  only.  A  major  problem  arose  for  the  study  group  here 
because  in  Package  3  there  were  three  new  major  equipment  categories  (antenna 
coupler,  digital  data  device,  COMSEC),  which  were  not  included  on  the  original 
work  Package  2  answer  sheet.  All  three  of  these  categories  were  missed  by 
almost  everyone  (see  Table  6-6).  Only  one  participant  broke  his  answers  into 
generic  and  reference  categories.  Even  the  HCM  designers’  answers  were  not 
broken  down  this  way.  The  HCM  contractor  did  not  use  the  workload  worksheets 
for  his  answers  even  though  they  were  provided  to  the  participants.  Only  one 
participant  used  the  appropriate  worksheet  from  the  methodology.  This  had 
been  done  in  the  training  course  but  was  not  a  requirement  of  the  replication. 
The  level  of  indenture  was  again  a  problem  for  the  study  group.  Everyone  was 
missing  a“  least  some  detail. 

The  HCM  designers  pointed  out  during  the  discrepancies  review  that  this 
type  of  effort  would  typically  be  a  multi-person  effort  and  that  missed  areas 
could  be  picked  up  within  later  data  packages.  Almost  all  of  the  participants 
felt  that  either  not  enough  detail  was  provided  or  the  solution  did  not 
logically  follow  from  the  methodology  instructions.  Upon  clarification  of  the 
problems  within  this  element  of  the  replication,  it  appeared  that  the  par¬ 
ticipants  had  a  reasonable  understanding  of  how  to  apply  tasks  to  equipment. 

A.  Work  Package  A  -  Reliability  and  Maintainability  (R&M) 
Determination 

In  this  package  the  analysts  were  directed  to  use  given  input  data 
and  then  to  perform  a  reliability  and  maintainability  analysis  of  the  SINCGARS 
system  manpack  configuration.  The  final  result  of  this  effort  was  the  deter¬ 
mination  of  the  system  equipment  maintenance  workloads.  The  participants  were 


Table  6-6.  Work  Package  3  -  Respondent  vs.  Dynamics  Research 
Corporation  Results 


Major 

Equipment  Categories 


_ Respondents _  Number 

123456  Correct 


Manpack 

Receiver  Transmitter 


X  X  X  X  X  X 


Antenna  Coupler 
ECCM  Unit 


X  X  X  X  X  X 


Digital  Data  Device 


COMSEC 


X  X  X  X  X  X 


to  take  the  tasks  and  equipment  on  the  workload  worksheets,  perform  the  cal¬ 
culations  as  specified  in  the  instructions,  and  ultimately  derive  maintenance 
ratios  (see  Table  6-7,  column  14,  for  a  sample  workload  worksheet).  This  was 
to  be  done  for  three  types  of  systems:  a  reference  system  and  two  proposed 
systems,  one  from  Cincinnati  Electronics  (CE)  and  one  from  ITT.  This  exercise 
resulted  in  an  examination  of  approximately  300  maintenance  ratios  for  each 
participant.  Unfortunately,  wrong  frequency  information  was  given  in  the 
instructions  for  using  operating  hours.  The  wrong  usage  metric  (annual 
operating  hours)  was  given  for  the  CE  ECCM  module.  The  contractor  answer 
sheets  had  several  problems.  General  and  direct  support  were  interchanged  for 
the  reference  system  ECCM  module.  CE  SRCU  answers  were  given  even  though  the 
participants  were  only  supposed  to  use  reference  system  data.  A  few  minor 
components  (e.g.,  battery)  were  missing.  Most  importantly,  several  answer 
sheets  were  wrong  and  had  to  be  replaced  by  corrected  ones. 

In  view  of  the  above  problems,  it  was  decided  to  grade  the  study  group 
based  on  what  they  thought  was  the  proper  procedure.  The  use  of  reference 
system  data  for  components  of  the  CE  and  ITT  were  not  included.  Because  one 
participant  (1)  did  contact  the  designers  and  obtain  the  correct  CE  ECCM  usage 
metric,  the  use  of  the  incorrectly  given  value  was  counted  as  wrong.  Table  6-8 
gives  the  percentage  of  correct  answers  with  clarifying  comments.  The  use  of 
the  correct  usage  metric  for  the  CE  proposed  system  is  obvious  for  Respondent 
1.  His  percentage  correct  was  94  while  everyone  else  varied  between  59  and 
71.  Several  respondents  (3,4,5)  were  missing  some  system  components.  Par¬ 
ticipant  6  disregared  some  columns  on  the  workload  worksheet  but  still  managed 
to  obtain  high  scores.  Similarly,  Participant  3  seemed  to  differ  from  the  HCM 
designers  in  his  approach  but  still  arrived  at  correct  answers. 


Table  6-8.  Work  Package  4  -  Respondent  vs.  Dynamics  Research 
Corporation  Results 
(%  Correct-Maintenance  Ratios) 


Respondent 

Reference 

CE 

ITT 

Comments 

1 

70 

94 

87 

E 

2 

95 

61 

94 

A 

3 

95 

59 

82 

A,  B,D 

4 

40 

60 

95 

A,B,E 

5 

50 

71 

89 

A,B 

6 

100 

71 

89 

C 

X 

75 

69 

89 

A  -  Wrong  usage  metric  for  CE 
B  -  Missing  items  -  Reference,  ITT 
C  -  Missing  calculation  steps 
D  -  Different  calculation  procedure  than  DRC 
E  -  Wrong  total  actions  -  ITT,  Ref 


The  study  group  had  several  comments  on  the  incorrect  and  confusing 
replication  instructions.  Most  commented  on  the  extensive  calculations,  time 
limitations,  and  tedium  caused  by  the  task.  One  participant  (3)  resorted  to 
using  a  programmable  calculator. 

During  the  discrepancy  re.iew,  the  HCM  designers  noted  that  this 
procedure  has  now  been  simplified  by  using  updated  LSA  data  sheets  and  by 
automating  the  worksheet. 

Despite  all  the  problems,  the  number  of  answers  that  matched  the  con¬ 
tractor's  calculations  were  relatively  high  (generally  between  60  and  95%). 
This  indicated  that  at  least  for  SINCGARS,  the  study  group  probably  understood 
how  to  apply  this  phase  of  HARDMAN. 


5.  Work  Package  5  -  Design  Difference  Index  (DDl) 

The  participants  were  directed  to  develop  the  SINCGARS  proposed 
design  in  the  manpack  mode  for  the  two  contractors  (CE  and  ITT)  by  evaluating 
and  compiling  a  list  of  reference  system  versus  proposed  system  design  dif¬ 
ferences.  The  study  group  was  then  supposed  to  determine  the  impacts  on 
system  workload  of  each  design  difference.  The  resultant  perturbation  values 
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(PV)  were  to  be  used  for  adjusting  the  reference  system  design  so  that  equit¬ 
able  comparisons  could  be  made  between  the  reference  and  proposed  systems. 

This  task  generated  the  most  discussion  during  the  discrepancy  review. 

The  discussion  centered  on  the  nature  of  the  comparisons.  For  instance,  should 
a  reference  system  be  used  "as  is"  or  should  there  be  an  adjusted  reference 
system  if  reference  and  proposed  systems  have  significantly  different  tech¬ 
nologies  and  time  frames.  This  question  was  not  addressed  during  the  training 
course  or  the  replication.  Because  this  problem  was  discussed  earlier  in 
Section  V,  it  will  not  be  pursued  here. 

As  in  packages  1  and  3,  the  work  that  Package  5  answers  were  quite 
subjective  (see  Table  6-9  for  a  sample  contractor  answer  sheet).  This  package 
had  the  highest  variance  among  participant  answers.  The  contractor  answers 
had  only  two  PVs.  Participant  3  had  30  values.  Individual  6  had  16.  Par¬ 
ticipants  4  and  5  had  none.  Participant  5  left  out  the  ECCM,  ECCM  Fill,  and 
SRCU  subsystems.  Individuals  4,  5,  and  6  made  no  reference  to  the  LSA-02 
reference  data. 

It  was  found  in  the  debriefing  that  the  study  group  had  trouble 
determining  the  design  differences  and  did  not  understand  the  use  of  the  PV. 

In  summary,  it  is  apparent  from  the  study  group  feedback  and  work  package 
answers,  that  this  step  could  not  be  closely  replicated  and  further 
clarification  is  needed. 


6.  Work  Package  6  -  Select  and  Assign  MOS 

The  objective  in  this  part  of  the  replication  was  to  record  the 
MOS  and  skill  levels  that  are  currently  performing  each  operation  and 
maintenance  task  for  the  reference  system  equipment  in  column  6  of  the 
workload  worksheet.  MOS  and  skill  levels  were  also  to  be  recorded  for  the 
contractor  proposed  systems.  Lastly,  for  those  tasks  omitted  by  the 
contractor,  the  reference  system  data  were  to  be  used. 

Because  the  study  group  had  previously  calculated  the  column  14 
maintenance  ratios  on  the  workload  worksheets,  it  is  likely  that  the 
replication  effort  would  have  proceeded  more  easily  if  this  task  had  been  done 
earlier.  This  was  reiterated  during  the  discrepancy  review.  Additionally,  it 
was  pointed  out  that  the  MOS  selection  process  needs  additional  refinement  by 
the  HCM  designers.  Presently  there  are  no  systematic  decision  procedures  for 
deciding  which  tasks  are  to  be  matched  with  the  appropriate  MOS  and  skill 
levels. 

Once  again,  there  were  problems  with  the  given  answers.  The  grader  was 
not  given  all  the  answer  sheets.  LSA  answers  were  supplied  for  components 
where  reference  data  should  have  been  used.  Lastly,  CE  R/T  answers  were  given 
when  the  study  group  had  not  been  given  the  inputs.  As  opposed  to  Package  4, 
the  reference  components  jointly  included  in  the  proposed  systems  were 
included  in  the  grading.  This  inclusion  is  not  very  significant  (6  out  of  151 
answers  for  the  CE  (4%)  and  13  of  141  for  the  ITT  (9%)),  but  it  was  done  here 
because  the  participants  were  directed  to  do  so  in  the  instructions. 
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All  participants  primarily  had  problems  with,  or  could  not  understand, 
the  contractor  MOS  selection  process.  In  fact,  one  individual  used  pre¬ 
decessor  system  MOS  instead  of  those  of  the  reference  system.  Even  though  it 
was  only  necessary  to  copy  MOS  from  the  correct  data  sheets,  the  average  scores 
were  not  particularly  high  (55-80%)  (Table  6-10).  Two  possible  contributing 
factors  are  (1)  the  recurring  problems  with  the  methodology  instructions  and 
(2)  the  boredom  caused  by  the  tedious  nature  of  filling  in  approximately  300  MOS 
and  skill  levels. 


7.  Data  Package  7  -  System/Component  Usage  Rates 

In  this  package  the  usage  rates  for  the  components  of  SINCGARS 
were  to  be  determined.  This  data  is  required  for  column  15,  usage  rate,  of 
the  workload  worksheets  (see  Table  6-7).  Because  only  one  value  for  each 
component  (R/T,  ECCM,  ECCM  Fill,  SRCU)  was  required,  this  was  the  most 
straightforward  assignment  of  the  entire  replication  effort.  The  entire  study 
group  had  the  correct  answers.  Note  that  in  doing  a  firsthand  application  in 
the  real-weapon  development  environment,  usage  data  might  be  difficult  to 
acquire. 


Table  6-10.  Work  Package  6  -  Dynamics  Research  Corporation 
vs.  Respondent  Answers  (%  Correct  MOS  Answers) 


Respondent 

Reference 

CE 

ITT 

Comments 

1 

50 

95 

79 

A 

2 

77 

87 

73 

C 

3 

73 

79 

67 

C 

4 

50 

97 

63 

C,D 

5 

45 

44 

91 

A,C,D 

6 

36 

85 

69 

B,  D 

X 

55  81  74 

A 

-  Missing 

Reference  System  R/T  MOS. 

B 

-  Assumed 

DRC  used  predecessor  rather  than  reference. 

C 

-  Missing 

CE  or  ITT  R/T  MOS. 

D 

Missing 

Reference  Components  for  ITT. 
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8.  Work  Package  8  -  Determine  Workload  and  Manpower  Requirements 

The  goal  of  this  final  package  was  to  determine  the  workload 
results  and  manpower  requirements  for  the  reference  and  proposed  systems.  The 
workload  results  were  obtained  by  multiplying  the  maintenance  ratios  from  Work 
Package  4  by  the  scenario  usage  rate  from  Package  7.  That  result  was  to  be 
multiplied  by  the  indirect  productivity  factor  (IPF)  to  obtain  the  workload 
results.  The  next  step  was  to  add  the  workloads,  by  component,  to  get  the 
total.  The  manpower  requirements  were  to  be  obtained  by  multiplying  the 
workload  totals  by  one  of  two  manpower  availability  factors  that  were  supplied 
by  the  contractor.  Because  this  last  step  was  trivial,  only  workload  totals 
were  analyzed  in  detail. 

The  HCM  design  team  answers  had  several  shortcomings.  Calculation 
mistakes  were  found  that  had  to  be  corrected,  and  LSA  data  were  incorrectly 
used  in  place  of  reference  system  data.  With  respect  to  the  study  group,  the 
use  of  wrong  MOS  was  evident  for  five  respondents.  Reference  data  were  left 
out  by  four  of  the  participants.  Four  individuals  also  made  calculation 
mistakes.  The  participants  were  confused  by  the  contractor  justification  for 
selection  of  IPF  and  manpower  availability  factors.  Two  also  commented  on 
time  limitations.  Given  the  above  problems,  in  addition  to  the  problems  with 
the  previous  packages,  it  was  surprising  that  the  study  group  got  any  of 
Package  8  correct.  Actually,  the  average  percent  correct  was  between  25  and 
45  (Table  6-11).  One  individual  (2)  averaged  69%  correct  for  each  of  the 
three  systems. 


D.  QUALITATIVE  ACCURACY  STUDY 

The  names  of  pertinent  military  personnel  were  supplied  by  ARI  and  the 
Navy  HARDMAN  office.  These  contacts  covered  a  wide  range  of  Navy  and  Air 
Force  procurement  projects  that  required  manpower  and  training  forecasts 
(Table  6-12).  The  feedback  on  HARDMAN  with  respect  to  these  real-life 
applications  varied  from  highly  positive  to  negative.  It  was  noted  that  these 
impressions  could  be  at  least  somewhat  affected  by  the  personal  biases  of 
people  contacted.  Nevertheless,  as  a  rough  estimate  of  accuracy,  it  was  felt 
that  these  contacts  were  a  good  initial  step. 

The  Advanced  Lightweight  Torpedo  (ALWT)  project  logistician  was  highly 
favorable  (33).  He  was  impressed  with  HARDMAN  in  that  its  initial  application 
could  start  with  very  little  input  data.  Its  resulting  projections  appeared 
to  be  within  about  10%  of  actual  manpower  data.  His  immediate  experience 
applied  to  the  reliability  and  maintainability  areas.  He  also  pointed  out 
that  these  estimates  went  through  the  Navy's  Logistics  Review  Council  (LRC) 
unscathed.  This  typically  happens  only  thirty  percent  of  the  time.  Most  of 
those  projects  that  do  not  pass  have  unsatisfactory  manpower  and  training 
estimates.  HARDMAN  also  saved  ALWT  large  sums  of  money  because  of  its  ability 
to  highlight  possible  problems.  These  comments  were  reiterated  by  the  ALWT 
ILS  manager  although  he  added  that  the  HCM  was  difficult  to  understand  (34). 

He  also  pointed  out  that  HARDMAN  provides  a  consolidated  data  base  that 
facilitates  sensitivity  analyses.  The  ALWT  and  SIRCS  MPT  logistic  element 
manager  felt  that  once  the  total  integrated  methodology  (HARDMAN  plus  training 
requirements  determination  plus  life-cycle  costing)  is  developed,  it  will  be 


Table  6-11.  Work  Package  8  -  Dynamics  Research  Corporation 
vs.  Respondent  Results 
Z  Workload  Results  Correct 


Table  6-12.  Qualitative  Accuracy  Study  Contacts 


Project 

Contact 

SUBACTS 

-  Submarine  Combat  System 

Training  Manager 

ALWT 

-  Advanced  Lightweight  Torpedo 

ALWT  Logistician 

ALWT 

-  Advanced  Lightweight  Torpedo 

ALWT  ILS  Manager 

DDGX 

-  Destroyer 

Personnel  and  Training 
Analyst 

ALWT 

-  Advanced  Lightweight  Torpedo 

MPT  Logistics  Element 
Manager 

SIRCS 

-  Shipboard  Intermediate  Range  Combat 

System 

MPT  Logistics  Element 
Manager 

VTXTX 

-  Flight  Trainer 

Project  Manager 

Air  Force 

-  ASSET  and  other  similar 

ASSET  Operations  Research 

techniques 

Analyst 

very  useful  for  the  early  identification  of  manpower  requirements  (35).  In 
comparison  with  other  similar  methodologies  he  felt  that  HARDMAN  was  less 
cumbersome  and  more  accurate.  He  also  pointed  out  that  one  drawback  of  this 
type  of  methodology  is  that  it  does  not  interface  with  hardware  design 
requirements . 

The  training  manager  for  the  SUBACTS  program  felt  that,  although  the 
HARDMAN  manpower  forecasts  have  been  fairly  accurate,  there  were  problems  in 
understanding  the  approach  (36).  The  SUBACTS  program  brought  in  an  independent 
contractor  to  use  HARDMAN,  which  helped  make  the  approach  more  understandable. 
Concurrently,  the  new  projections  proved  to  be  as  accurate  as  the  original 
forecasts. 

One  of  the  DDGX  destroyer  training  analysts  has  had  MPT  experience  on 
pilot  programs  within  NAVC,  NAVAIR,  and  NAVLX.  He  stated  that  there  was  no 
standard,  overall  methodology  for  manpower  estimates  at  present  although  it  is 
planned  (by  January  1984)  to  evaluate  a  Navy  program  (SYSCOM)  with  the  help  of 
the  HARDMAN  office.  He  felt  that,  although  total  feedback  on  HARDMAN  has  not 
been  received  as  yet,  it  is  probably  quite  useful  and  will  have  many 
applications  within  the  Navy  (37). 

The  project  manager  for  the  VTXTX  flight  trainer  was  too  far  removed 
from  forecasting  to  say  how  closely  HARDMAN  resembled  real  life.  With  respect 
to  other  similar  methodologies,  he  stated  that  it  has  typically  been  difficult 
to  match  their  outputs  to  actual  performance.  This  has  usually  been  due  to 
the  differing  ways  the  methodologies  define  system  components  as  compared  to 
the  actual  system  design  (38). 

An  operations  research  analyst  at  the  Air  Force's  Human  Resources 
Laboratory  has  been  involved  with  several  Air  Force  projects.  She  has  worked 
with  HARDMAN-type  methodologies  (e.g.,  coordinated  Human  Resources  Technology, 
ASSET)  but  has  found  to  date  they  have  not  been  widely  applied  (39). 


E.  RESULTS  OF  MANPOWER  SENSITIVITY  ANALYSIS 

The  problem  explored  in  this  portion  of  the  reliability  analysis  was  how 
accurate  manning  estimates  must  be  in  a  new  system  acquisition.  Because  the 
penalty  and  cost  of  estimation  can  be  quite  high,  it  is  important  to  be  clear 
about  the  benefits  of  accuracy  and  the  costs  of  inaccuracy,  particularly  when 
developing  a  projection  technique  such  as  the  HCM.  Further,  because  manning 
is  not  estimated  in  a  vacuum,  it  is  important  to  establish  whether  concentra¬ 
tion  on  accuracy  in  other  related  variables  will  have  any  impact  on  the 
manning  estimation. 

In  this  portion  of  the  analysis,  permissable  error  bounds  were 
determined  and  applied  to  the  SINCGARS  program  to  study  the  interplay  of 
manning  estimate  accuracy  and  the  accuracy  of  other  variables.  Error  bounds 
were  determined  through  interviews  with  decision  makers  involved  in  program 
review  offices  such  as  the  Office  of  the  Secretary  of  Defense  (OSD)  or  the 
Cost  Analysis  Investigation  Group  (CAIG).  For  this  small  study,  most  of  the 
individuals  contacted  were  associated  with  OSD.  Data  descriptive  of  the 
SINCGARS  system  were  obtained  from  the  SINCGARS  program  office.  Error 


relationships  were  investigated  through  the  use  of  an  interactive  life-cycle 
cost  model  developed  for  the  U.  S.  Navy  and  adapted  to  this  particular 
exercise  (8). 


The  overall  conclusion  of  this  brief  study  is  that  accuracy  of  manning 
estimates  may  not  be  a  significant  issue  in  the  development  of  tools  and 
methods  for  acquisition  support  and  cost  analysis.  For  example ,  if  manning 
estimates  were  scrutinized  by  the  budget  controllers  and  were  found  to  be  as 
poor  as  the  15%  boundaries  allow,  the  error  in  spare  stockage  costs  (a  major 
LCC  variable)  would  be  almost  50%.  Therefore,  budget  controllers  would  most 
likely  flag  other  important  quantities  long  before  the  manning  estimates  would 
exceed  their  permissable  limits.  This  conclusion  is  explored  in  greater 
detail  in  the  following  paragraphs. 


1.  Permissable  Error  Criteria 

The  costs  of  error  vary  by  circumstance  and  direction.  Many 
believe,  for  example,  that  the  ‘'cost  growth"  phenomenon  marking  system 
acquisitions  of  the  past  decades  is  partly  explained  by  poor  cost  estimating 
in  the  early  stages  of  acquisition.  Nonetheless,  this  facet  of  many  programs 
has  been  a  contributing  factor  to  cancellation. 

A  theoretician  would  relate  the  permissable  range  of  error  to  the 
trade-off  between  the  cost  of  error  and  the  cost  of  reducing  it.  In  the  case 
of  manning  estimation  error,  the  major  variables  have  to  do  with  establishing 
appropriate  personnel  pipeline  flows  at  the  right  times  to  make  required 
resources  available  on  time.  Conversely,  excess  flows  are  to  be  avoided. 

When  these  flows  are  of  the  wrong  size,  costs  are  incurred  in  adjusting  them. 
Costs  of  concern  to  the  Department  of  Defense  come  in  two  forms:  dollars  and 
readiness.  These  are  looked  upon  differently  by  decision  makers. 

OSD  (or  CAIG)  personnel  responsible  for  questioning  the  estimates  of 
cost  and  support  requirements  for  new  systems  look  for  (1)  gross  error  in  cost 
estimates  and  (2)  errors  in  readiness-critical  quantities.  If  total  dollars 
is  the  only  issue,  an  estimated  error  of  about  $50  million  (in  an  estimated 
quantity)  seems  to  be  required  before  decision  makers  are  willing  to  slow  down 
the  progress  of  a  program  to  check  for  more  accurate  estimates.  By  the  same 
token,  very  small  amounts  of  dollars  may  be  involved  in  such  decisions  if  the 
variable  being  estimated  is  critical  to  readiness .  Here  the  rule  is  that  the 
cost  estimate  may  not  be  off  by  more  than  about  15%,  regardless  of  how  small 
the  total  dollar  amount  is. 

The  mechanism  by  which  these  limits  are  employed  is  the  application  of 
experience  and  judgment  to  assess  cost  projections  from  program  offices.  The 
decision  makers  queried  have  all  studied  the  relevant  issues  for  many  years  on 
a  variety  of  systems.  It  was  assumed  that  their  experience  provided  them  with 
a  reasonable,  independent  assessment  of  the  value  of  specific  quantities  of 
interest.  If  their  assessments  differ  from  those  presented  by  program  offices 
by  more  than  the  amounts  indicated  above,  this  gives  rise  to  a  question  and 
request  for  further  analysis. 

To  formalize  this  mechanism,  it  can  be  compared  to  a  statistical  test  of 
hypothesis.  The  decision  maker's  estimate  is  the  population  mean  and  the 
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program  office  submission  is  Che  sample  mean.  The  decision  maker  acts  as  if  a 
15%  difference  is  sufficient  to  reject  the  hypothesis  that  the  sample  is  drawn 
from  the  same  population. 


2.  Results  of  the  Analysis 

The  procedure  followed  was  to  drive  manning  to  +15%  of  its  value 
in  the  baseline  estimate  by  varying  several  different  input  values.  The 
results  for  MTTR,  MTRR,  MTBF  and  usage  rate  are  shown  in  Figure  6-1.  The 
figure  shows  the  positive  error  boundary  for  manning  on  the  vertical  axis  as  a 
function  of  various  LCC  input  values  on  the  horizontal  axis.  The  closer  the 
plot  of  each  variable  is  to  the  horizontal  axis,  the  less  responsive  manning 
is  to  variations  in  that  variable.  Reading  the  figure,  the  farther  to  the 
right  a  line  intersects  the  +15%  line,  the  greater  the  error  in  that  variable 
necessary  to  produce  the  fixed  error  in  manning. 

The  most  powerful  variable  is  usage  rate  because  this  simply  acts  as  a 
scalar  against  all  demand  rates.  Failure  rate  (the  inverse  of  mean  time 
between  failure)  is  the  next  most  powerful  determinant,  driving  manning  to  its 
lower  boundary  at  half  its  baseline  value.  The  two  maintainability  variables 
are  the  least  powerful,  mean  time  to  remove  and  replace  having  a  greater 
effect  than  mean  time  to  repair.  This  is  explained  by  the  fact  that  in  the 
basic  data  set,  the  latter  is  only  one  hour  while  the  former  is  three  hours. 
This  difference  is  accentuated  by  the  fact  that  four  of  the  LRU  are  coded 
discard  at  failure,  which  means  that  they  will  be  removed  and  replaced  but  not 
repaired. 

Figure  6-2  is  a  plot  of  life-cycle  cost  variation  against  manning  levels 
produced  by  changes  in  the  four  major  variables.  It  is  clear  that  usage  rates 
have  a  strong  effect  throughout  the  cost  estimate  while  reliability  and 
maintainability  statistics  have  a  much  smaller  influence.  Notice  also  that 
mean  time  to  repair,  even  when  driven  to  zero,  cannot  produce  the  -15%  manning 
error. 


Another  comparison  of  value  is  the  different  effect  of  MTBF  errors  on 
manning  and  spares.  The  error  bounds  on  spares  are  reached  at  86.7%  and 
118.5%  of  baseline  MTBF.  The  corresponding  manning  error,  however,  is  only 
4.7%  and  -4.6%  at  these  limits.  To  reach  the  _+15%  manning  errors,  the 
multiples  required  are  66.8%  and  200%  of  baseline  MTBF.  These  relationships 
are  illustrated  in  Figure  6-3.  Thus,  it  appears  that  for  this  particular 
case,  reasonable  scrutiny  of  the  spares  cost  estimate  (and  hence  reliability) 
would  be  likely  to  confine  the  value  of  the  manning  error  well  within 
prescribed  limits.  Similar  comparisons  cannot  be  made  for  the  maintainability 
variables  because  they  do  not  affect  readiness-critical  outputs  other  than 
manning  and  training. 

The  remaining  possible  source  of  excess  error  in  manning  estimates  is 
from  joint  errors  in  input  variables.  This  problem  can  be  illustrated  by 
joint  errors  in  maintainability  and  reliability  data.  MTRR  is  used  for  the 
former  and  MTBF  for  the  latter. 

Figure  6-4  illustrates  the  accelerated  effect  of  joint  errors  in  MTBF 
and  MTRR  in  the  presence  of  errors  in  the  manning  variable.  The  two  graphs 
are  the  locus  of  manning  errors  produced  by  different  levels  of  error  in  MTRR, 
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Figure  6-1.  Errors  in  Input  Variables  Producing  15%  Error  in  Manning 


LIFE-CYCLE  COST, 


MANNING  ERROR,  %  deviation 


MTBF:  86.7%  OF  BASELINE 

(SPARES  UPPER  BOUNDARY) 


118.5%  OF  BASELINE 
(SPARES  LOWER 
BOUNDARY) 


MANNING:  UPPER  LIMIT 


MANNING:  LOWER  LIMIT 


Figure  6-4.  Relationship  of  Joint  Errors  to  Manning 


given  a  certain  error  has  already  occurred  in  MTBF.  The  graphs  have  been 
drawn  for  those  values  of  MTBF  that  produced  the  upper  and  lower  boundary 
errors  in  spares  estimates. 

Notice  that  even  under  the  joint  error  condition,  a  relatively  large 
error  in  MTRR  is  required  to  produce  a  15%  error  in  manning.  For  an  MTBF 
(118.52  of  the  baseline)  that  produces  a  15%  underestimate  in  spares,  MTRR 
must  also  be  understated  by  almost  fifty  percent  (0.542)  of  its  true  value  to 
produce  a  manning  estimate  15%  lower  than  its  true  value.  Under  the  same  MTBF 
error  an  overestimate  of  87.6%  in  MTRR  is  required  to  produce  a  15%  overstate¬ 
ment  of  manning  requirement.  For  an  understated  mean  time  between  failure 
(86.72  of  the  baseline),  the  MTRR  must  be  misestimated  at  about  two  thirds  its 
value  (0.37)  and  almost  a  third  again  its  real  value  (133.9%)  to  produce  mann¬ 
ing  estimates  off  by  minus  and  plus  152,  respectively.  This  one  example  sug¬ 
gests  that  manning  may  not  be  a  particularly  sensitive  variable  in  comparison 
to  other  dependent  LCC  variables. 


F.  RELIABILITY  IMPLICATIONS 

1.  Internal  Reliability 

The  four  major  findings  of  the  SINCGARS  replication  were  as 

follows : 

(1)  There  were  problems  in  understanding  how  to  construct  the 
reference  system. 

(2)  There  was  confusion  over  the  selection  and  use  of  design  dif¬ 
ference  indexes,  perturbation  values,  and  indirect  productivity 
factors . 

(3)  There  was  a  lack  of  understanding  of  MOS/skil 1-level  determination 

(4)  It  was  apparent  that  this  type  of  analysis  should  be  performed  by 
a  multidisciplinary  team,  not  one  analyst. 

Among  the  causes  for  these  findings,  beyond  the  general  problems  of  lack 
of  methodology  clarity  and  inaccuracies  caused  by  not  having  the  technique 
automated,  were  problems  with  the  instructions,  insufficient  or  missing  supple 
mental  input  data,  and  excess  emphasis  on  calculations  instead  of  reasoned 
judgments.  It  was  also  apparent  that  the  replication  deviated  too  much  from 
the  training  course  content.  Additional  constraints  included  incorrect  answer 
sheets  upon  which  later  analyses  were  based,  syntax  differences  in  the  subjec¬ 
tive  work  packages,  and  the  tedium  brought  on  by  the  enormous  quantity  of 
answers  required  by  Work  Packages  4  and  5.  Interestingly,  the  average  number 
of  answers  that  matched  the  HCM  design  team  was  high  (78%  in  Package  4,  70%  in 
Package  6)  in  some  of  the  packages.  However,  in  the  all-important  Data  Pack¬ 
age  8,  only  32%  of  the  answers  matched  the  corrected  contractor  response. 

This  is  a  function  of  both  the  HCM  clarity  problem  and  the  snowballing  effect 
of  previous  administrative  irregularities  already  present  in  the  inputs  of 
Package  8.  Even  with  all  the  problems  previously  discussed,  the  study  group 
appeared  to  understand  how  to  apply  HARDMAN  to  SINCGARS  within  the  regulated 


atmosphere  of  the  replication.  This  finding  was  derived  from  the  reasonably 
good  performance  on  roughly  half  the  data  packages  and  the  apparent  improve¬ 
ment  in  methodology  comprehension,  once  the  discrepancies  review  conference 
was  held.  It  is  probably  only  now  that  the  designers  understand  all  Of  the 
problems  that  existed  within  the  replication.  If  the  original  replication 
were  used  as  a  pilot  study,  a  comprehensive  and  consistent  reliability  check 
could  be  performed. 


2.  Qualitative  Accuracy  Study 

It  seems  that  some  of  military  contacts  who  deal  with  manpower  and 
training  have  somewhat  consistent  opinions  with  the  results  of  the  face 
reliability  and  replication  analyses.  That  is,  the  methodology  is  complicated 
and  hard  to  understand,  but  when  understood  and  applied  correctly,  has  the 
potential  to  provide  reasonable  manpower  and  training  forecasts  for  new 
procurements.  This  difficulty  in  understanding  HARDMAN  negatively  influenced 
some  of  the  personnel  queried,  or  else  others  surmounted  this  obstacle  and 
were  able  to  make  good  use  of  the  technique.  It  can  be  conjectured  here  that 
the  comprehension  difficulty  experienced  by  the  personnel  contacted  is  similar 
to  the  problems  encountered  during  the  replication  experiment.  Once  the 
methodological  loose  ends  are  corrected,  it  appears  that  HARDMAN  could  have 
widespread  acceptance. 


3.  Manpower  Sensitivity  Study 

The  sensitivity  analysis  suggests  that  manning  estimates  may  be 
fairly  robust;  they  may  not  be  as  responsive  to  errors  in  input  variables  as 
the  other  important  LCC  elements.  Because  several  of  these  quantities,  such 
as  MTTR  and  MTBF,  are  scrutinized  for  accuracy  during  the  review  process,  and 
because  they  are  more  responsive,  it  appears  likely  that  errors  in  inputs  will 
be  detected  through  them  before  they  drive  manning  estimates  out  of  the 
permissable  boundaries. 

With  regard  to  the  data  that  drive  these  estimates,  the  results  suggest 
that  it  would  be  unwise  to  devote  extensive  resources  to  their  refinement  for 
the  purpose  of  improving  manning  estimates.  The  more  serious  sources  of 
significant  error  in  manning  estimates  that  can  be  isolated  from  this  study 
are  the  inappropriate  selection  of  an  operating  scenario  or  deployment  plan. 
The  TRADOC  and  DARCOM  communities  control  cost  and  manning  estimates  to  a  far 
greater  extent  than  normally  realized.  By  setting  the  number  to  be  purchased, 
the  number  of  operating  units,  and  the  hours  of  operation  for  the  equipment, 
the  program  offices  have  the  largest  single  effect  on  estimated  quantities  of 
anyone  in  the  design  process.  Because  early  in  the  acquisition  process  these 
values  are  frequently  not  known  with  any  precision,  they  are  often  considered 
unimportant  when,  in  fact,  they  are  pivotal  LCC  elements.  For  example,  a 
system  of  which  100  will  be  purchased  should  be  designed  and  built  differently 
from  one  where  10,000  will  be  built.  As  a  second  example,  usage  rate  dif¬ 
ferences  of  plus  and  +  13%  will  produce  error  in  manning  estimates  +15%. 

These  can  be  produced  through  an  error  in  the  estimated  hours  of  opera¬ 
tion  per  year  or  in  the  number  of  units  to  be  purchased.  If  both  are  off 
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by  6.3%  or  more,  then  the  joint  effect  on  usage  rate  will  be  15%  or  more  and 
the  manning  error  will  exceed  the  permissable  limits.  These  remarks  should  be 
taken  as  a  caution  to  program  offices  to  devote  the  time  and  energy  necessary 
to  establish  reasonable,  stable  values  for  these  variables.  Understanding 
that  the  sensitivity  analysis  provided  here  is  somewhat  limited  by  (1)  the 
sample  size  and  variety  of  acquisitions  studied  and  (2)  assumptions  concerning 
the  use  of  Navy  billet  data  as  representive  of  the  Army  billet  structure,  the 
findings  are  significant  enough  to  warrant  repeating  the  analysis  once  the 
Army  has  its  own  billet  costing  system  completed. 
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SECTION  VII 


CONCLUSIONS  AND  RECOMMENDATIONS 


A.  OVERVIEW 

The  preceding  sections  provided  the  findings  in  the  four  major  validity 
areas  of  face,  internal  reliability,  event-series,  and  qualitative  accuracy. 

At  the  end  of  each  section,  the  validity  implications  were  assessed,  along 
with  supporting  rationale.  In  this  last  section,  the  findings  are  first 
pooled  and  then  summarized  to  illustrate  the  strengths  and  common  major 
weaknesses  that  became  apparent  after  examining  and  comparing  all  the  validity 
implications.  Next,  the  resolution  of  these  common  problem  areas  is  addressed 
within  the  context  of  (1)  the  ranking  of  the  weaknesses  as  a  function  of  the 
user  requirements  and  their  impacts  on  the  accuracy  of  the  methodology;  (2) 
the  practical  aspects  of  improving  weak  areas  in  terms  of  difficulty  and 
potential  for  other  organizations  besides  the  original  designer  to  adopt  the 
technique;  and  (3)  the  reasonably  low  requirement  to  have  extremely  accurate 
MPT  projections.  Ultimately,  a  condensed,  prioritized  list  of  weaknesses  and 
recommended  corrective  actions  is  provided.  Additionally,  follow-on  study 
areas  deemed  important  to  closing  out  issues  that  could  not  be  addressed 
within  the  time  and  budget  constraints  of  this  evaluation  are  also  recommended. 


B.  SUMMARY  OF  RESULTS 

In  total,  the  HCM  proved  to  be  reasonably  sound.  Table  7-1  summarizes 
the  strengths.  The  net  strengths  were  based  primarily  on  the  high  degree  of 
conformance  with  the  user  requirements,  the  reasonable  similarity  to  other  MPT 
modeling  processes,  and  the  closure  of  eighty-five  percent  of  the  audit 
issues.  It  was  also  promising  to  discover  that  the  projections  did  not  have 
to  be  extremely  accurate,  considering  the  remaining  twelve  to  fourteen  problem 
areas  that  were  found  to  potentially  affect  the  accuracy  and  credibility  of 
the  projections.  The  ->otential  concerns  or  problem  areas  uncovered,  which 
generally  spanned  all  the  user  requirements,  operational,  and  reliability 
analyses,  are  shown  in  Table  7-2. 


C.  PRIORITIZATION  OF  VALIDITY  ISSUES 

Before  any  recommended  revisions  to  the  methodology  can  be  made,  it  is 
essential  to  examine  each  of  the  issues  from  Table  7-2  and  weigh  their 
importance  to  meeting  the  Army's  MPT  needs  and  their  implications  relative  to 
effecting  the  accuracy  and  usefulness  of  the  projections.  It  is  important  to 
recognize  in  the  near  term  that  the  methodology  is  reasonably  sound.  It  is 
equally  important  to  know  the  major  limitations  and  the  supporting  fact  that 
allowances  can  temporarily  be  made  for  some  of  the  limitations  in  light  of  the 
inherent  acceptable  error  bounds  on  the  manpower  projection.  For  the  near  and 
intermediate  terms  it  would  be  useful  to  improve  those  areas  most  important 
and  amenable  to  immediate  correction,  followed  by  a  plan  for  improving  the 
remaining  far-term  areas,  perhaps  requiring  more  supporting  data. 
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Table  7-1.  Net  Results  of  HARDMAN  Evaluation 


Evaluation  Area 

Net  Results 

User  Requirements 

Technique  complied  with  majority  of 
mandatory  user  needs 

Operational  Analysis 
Comparison  with  other 

MPT  methods 

Methodology  conformed  with  other  known, 
accepted  MPT  modeling  schemes  and  data 
foundations 

Audits 

Methodology  demonstrated  sound  logic  and 
reasonable  results  for  85%  of  test  issues 
examined  (remaining  issues  considered 
reparable  in  near  term) 

Reliability  Analysis 

Internal  reliability 

Test  group  correctly  replicated  half  of 
selected  test  points  (two  of  four  remaining 
test  points  found  to  be  repeatable  after 
clarification) 

Qualitative 

Accuracy  check 

Two  of  nine  individuals  having  experience 
with  HARDMAN  applications  indicated  a  rough 
accuracy  of  80-90%  with  actual  manning 
requirements;  remaining  individuals  had 
insufficient  experience  to  comment 

Event  Series  Validity 

Manpower  variable  found  to  be  least 
sensitive  of  all  life  cycle  cost  variables, 
demonstrating  good  utility  of  existing 
methodology  even  without  near-term 
improvements 
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Methodology  Step 


Validity  Issues 


Step  1  Data  base  construction 
and  functional  requirements 


Step  2  Build  reference  system 
and  establish  manpower/ 
skill  requirements 


Step  3  Determine  training 
requirements 


Steps  4  and  5  Determine 
personnel  requirements/ 
shortages 

Step  6  Perform  tradeoff 

analyses  and  present  results 


Overall  methodology 


No  firm  criteria  for  selecting,  altering, 
manipulating,  or  rating  data  sources 

Unclear  indication  of  functional- 
requirement  level  of  indenture 

No  firm  guidelines  for  constructing 
reference  system  (particularly  when  modeling 
two  widely  varying  proposed  designs) 

No  consideration  given  to  observed  reduced 
manpower  at  end  points  of  life  cycle 

No  firm  range  and  criteria  for  selecting 
design  difference  indices  (perturbation 
values),  induced  failure  factors,  or 
indirect  productivity  factors 

No  complete  means  provided  for  equally 
comparing  reference  and  proposed 
technologies 

Incomplete  guidelines  provided  for 
selecting  M0S/9kills 

No  quantitative  means  of  differentiating 
between  critical/non-critical  tasks  and 
resultant  impact  on  training 

Projection  does  not  consider  manpower 
available  at  actual  time  of  system 
deployment 

Unstructured  tradeoff  process 

No  indication  provided  for  either  relative 
credibility  of  projections  or  best 
estimate  for  manpower  planning 

Incomplete  life-cycle  cost  analysis 

No  standardized  assessment  of  study 
weaknesses  and  proposed  resolution  actions 

Low  usability  by  anyone  other  than  a 
well-seasoned,  multidisciplinary  team 


1. 


Near-term  Issues 


The  issues  having  both  sizeable  user  value  and  accuracy 
implications!  and  jointly  found  amenable  to  solution  within  the  immediate 
one-year  time  frame  were  as  follows: 

(1)  No  standardized  criteria  for  selecting,  altering,  manipulating,  or 
rating  data  sources. 

(2)  Lack  of  proper  level  of  indenture  for  functional  requirements. 

(3)  Insufficient  guidelines  for  constructing  reference  system. 

(4)  No  consideration  for  reduced  manpower  demands  at  system  life-cycle 
end  points. 

(5)  Unequal  comparison  of  perturbed  state-of-the-art  technology 
against  new  proposed  designs. 

(6)  Lack  of  guidelines  for  selecting  MOS  and  skills. 

(7)  Unstructured  tradeoff  process. 

(8)  No  credibility  indication  of  projections. 

(9)  Non-standardized  assessment  of  study  weaknesses  and  resolution 
actions. 

Logic  and  credibility  issues  such  as  items  (1),  (3),  and  (5)  above,  are 
probably  most  important  because  they  affect  both  the  accuracy  and  usefulness 
of  the  methodology.  Although  removal  of  these  problem  areas  would  not 
guarantee  improvement  in  absolute  accuracy,  it  would  certainly  provide  the 
users  with  a  better  subjective,  relative  indication  of  accuracy  to  aid 
decisions  on  budgeting  of  additional  hardware  tests  or  favoring  one  projection 
over  another  for  planning  purposes  until  additional  information  is  received. 

All  of  these  items  were  considered  candidates  for  immediate  resolution 
because  elements  of  the  solution  for  each  of  the  issues  often  already  existed 
in  both  the  HCM  design  team  and  the  Navy  concerning  the  relative  quality  of 
various  data  sources  (31,  40).  For  example,  actual  field  or  controlled 
collection  data  probably  ranks  highest  on  the  list  of  preferred  information 
for  modeling  new  system  performance  because  of  its  maturity,  followed  by 
operational  or  field-test  data  (reasonably  mature),  development  test  data,  and 
free-flow  or  semi-controlled  data  collection  information  (limited  sample 
size);  and  finally,  contractor  estimates  (no  test  or  sample  data). 

As  with  the  inherent  experience  available  on  data  sources,  it  was 
discovered  that  solution  structures  similarly  existed  for  items  (3)  and  (5) 
above.  The  use  of  two  reference  systems  to  model  two  widely  varying 
contractor  designs  was  introduced  earlier  in  Section  V-F  (the  SINCGARS 
audit).  Because  the  structure  already  existed  for  perturbing  historical 
predecessor  component  data  to  fill  data  gaps  in  the  proposed  system,  it 
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appeared  chat  a  simple  extension  of  the  same  procedure  would  allow  a 
"top-level"  comparison  of  the  total  predecessor  and  proposed  systems  on  an 
equal  basis. 

The  next  near-term  priority  closely  associated  with  credibility  is  the 
resolution  of  potential  error  sources.  Items  (2),  (4),  and  (6)  serve  as 
potential  error  sources  and  therefore  have  accuracy  and  credibility 
implications.  As  indicated  in  the  preceding  paragraphs,  it  was  learned  that 
much  of  the  solution  framework  already  existed  within  the  joint  experience  of 
the  HCM  design  team  who  normally  perform  the  HARDMAN  analysis  (34).  The 
reduced  end-point  manpower  demand  issue,  item  (4),  was  considered  a  fairly 
simple  error  source  to  correct,  considering  that  the  deployment  period  and 
life  cycle  could  be  broken  down  into  yearly  time  periods  to  be  sensitive  to 
the  gradual  buildup  or  tapering  off  of  systems.  Item  (6)  above,  was  another 
near-term  issue  that  was  found  easily  rectified  based  on  knowledge  obtained  at 
both  the  audits  and  discrepancy  review. 

The  final  near-term  priority  area  is  the  resolution  of  some  of  the 
ambiguity  issues.  Items  (7)  and  (9)  are  somewhat  linked  together  and  are 
rated  as  the  last  priority  because  their  solutions  serve  more  to  improve  the 
clarity  of  the  process  and  projections  than  to  contribute  directly  to  accuracy 
and  credibiity.  Again,  because  the  solution  components  for  listing  all 
potential  tradeoffs,  weaknesses,  and  proposed  actions  already  existed  to 
varying  degrees,  it  was  felt  that  these  adjustments  to  the  methodology  could 
be  a  simple  enhancement  to  the  decision-making  quality  of  the  technique. 

2.  Intermediate-term  Issues 

Some  of  the  validity  issues  discovered  during  the  evaluation  were 
found  to  have  accuracy  implications,  but,  upon  closer  examination  during  the 
audits  and  SINCGARS  discrepancy  review,  were  also  found  to  be  not  amenable  to 
immediate  solution.  Although  some  data  and  models  were  available  and  appeared 
to  represent  means  of  closing  the  issues,  it  was  obvious  that  a  certain  amount 
of  additional  research  and  methodology  restructuring  were  required  and  would 
probably  consume  to  two  years  of  effort.  The  issues  from  Table  7-1  that 
are  relevant  to  this  intermediate  category  are  as  follows: 

(1)  No  firm  range  and  criteria  for  selecting  perturbation  values, 
induced  failure  factors,  or  indirect  productivity  factors. 

(2)  No  consideration  for  manpower  availability  at  the  projected  system 
deployment  date. 

(3)  Incomplete  life-cyle  cost  analysis. 

In  line  with  the  reasoning  provided  earlier,  items  (2)  and  (3)  above 
would  have  to  be  given  highest  priority  because  they  both  impact  the  accuracy 
and  credibility  of  the  projections.  For  both  of  these  issues  modeling 
processes,  introduced  earlier  in  Section  V,  have  already  been  developed  and 
with  slight  redesign  could  be  incorporated  in  the  existing  methodology.  Item 
(1)  is  fairly  important  because  it  could  represent  a  major  error  source.  Of 
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the  three  different  types  of  factors,  the  perturbation  value  is  probably  the 
most  crucial  element.  Some  information  presented  during  the  SINCGARS 
discrepancy  review  is  available  to  help  define  the  problem  of  determining  what 
can  reasonably  be  expected  regarding  improvements  in  new  technology 
performance.  The  induced  failure  and  indirect  productivity  factors  were 
considered  slightly  larger  problems  because  it  is  difficult  to  determine  when 
mistakes  in  selection  of  an  arbitrary  value  for  either  one  will  generate  a 
serious  error  in  the  outcome. 


3.  Far-term  Issues 

Some  of  the  validity  issues  uncovered  were  found  to  require  a 
substantial  amount  of  data  and  time  (probably  about  three  to  five  years)  to 
resolve  them.  In  most  cases  the  issues,  although  important,  only  surfaced  in 
either  the  user  requirements  or  the  operational  analysis  and  were  therefore  of 
a  singular  nature.  The  issues  fitting  in  this  category  were  as  follows: 

(1)  No  quantitative  differentiation  between  critical  and  non-critical 
tasks. 

(2)  Incomplete  consideration  of  demographic  data  such  as  age,  sex,  or 
education. 

(3)  No  consideration  of  the  manpower  impacts  of  other  emerging  systems. 

(4)  No  consideration  of  major  socioeconomic  fluctuations  on  manpower 
availability. 

Probably  the  largest  potential  error  source  is  item  (3)  because  the 
overall  acquisition  and  planning  process  for  new  systems  spans  several  years. 
Consequently,  new  concepts  being  approved  have  a  very  good  chance  of 
overlapping  each  other  in  their  respective  design  and  support  planning 
stages.  The  remaining  items  are  fairly  equivalent  in  their  level  of 
importance. 

Great  care  was  taken  in  the  preceding  paragraphs  to  refrain  from 
including  suggestions  for  improvements  to  concentrate  on  presenting  a  clear 
rationale  for  the  near-,  intermediate-,  and  far-term  ordering  of  the  various 
problem  areas.  Having  prioritized  all  the  various  validity  issues  as  a 
function  of  their  potential  for  weakening  the  methodology  and  ease  of 
solution,  the  final  step  is  to  explore  sensible  solutions. 


D.  RECOMMENDATIONS 

One  way  to  develop  a  plan  for  implementing  recommendations  is  to  first 
convert  each  issue  into  a  positive  proposed  action  and  then  summarize  the 
total  scope  of  intended  actions.  Table  7-3  first  provides  a  summary  plan  for 
improving  the  existing  methodology,  followed  by  a  detailed  discussion  of  each 
suggested  corrective  action. 
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Table  7-3.  Recommended  Plan  for  Improving  HARDMAN  Methodology 


Potential 

for  Improving  Methodology 

Prioritized  List 

Near-Term 

Intermediate 

Far-Term 

of  Recommendations 

High  Med. 

High  Med. 

High  Med. 

Establish  criteria  for  rating, 
selecting,  altering,  or 
manipulating  data 

Provide  indication  of  credibility 
of  projections 

Establish  top-level  method  for 
comparing  technology  of  total 
reference/baseline  and  proposed 
systems 

Provide  better  guidelines  for 
constructing  reference  systems 

Firm  up  guidelines  for  functional- 
requirements  level  of  indenture 

Provide  a  better  structure  for  MOS 
and  skill  selection 

Reduce  MACRIT  manpower  projection 
down  to  one-  or  two-year  projec¬ 
tions  to  be  sensitive  to  lower 
end-point  demands 

Provide  a  more  structured 
tradeoff  process 

Establish  standardized  assess¬ 
ment  of  study  weaknesses  and 
resolution  actions 


Project  manpower  availability 
out  to  actual  system  deployment 
date 

Provide  more  complete  lift-cycle 
cost/tradeoff  analysis 


Potential 

for  Improving  Methodology 

Prioritized  List 
of  Recommendations 

Near-Term 
High  Med. 

Intermediate 
High  Med. 

Far-Term 

High  Med. 

Firm  up  ranges  for  perturbation 
values,  induced  failure,  and 
indirect  productivity  factors 

X 

Incorporate  potential  manpower 
impacts  of  other  emerging 
systems 

X 

Provide  quantitative 
differentiation  betwen  critical 
and  non-critical  tasks 

X 

Incorporate  more  demographic 
data  into  personnel/availability 
projections 

X 

Consider  major  socioeconomic 
fluctuations  in  manpower 
availability  projection 

X 

The  following  paragraphs  develop  each  of  the  recommendations  in 
Table  7-3  by  suggesting  possible  corrective  actions.  First,  consideration  is 
given  to  the  methodology  design,  followed  respectively  by  suggested  training 
for  potential  HARDMAN  analysts  as  well  as  follow-on  research. 


1.  Methodology  Design 

Because  the  key  credibility  issues  were  primarily  centered  in  the 
near-  and  intermedite-term  time  frames,  the  following  discussion  focuses  on 
those  recommendations  that  fit  in  that  one-  to  two-year  time  span.  Far-term 
corrective  actions  are  discussed  in  the  "follow-on"  portion  of  this  subsection. 


a.  Suggested  Corrective  Actions  for  Data  Management  and 
Credibility  Indication.  It  is  generally  the  intent  of  any  modeling  technique 
to  act  as  a  guide  or  tool  to  assist  decision  makers.  As  suggested  earlier  in 
Section  IV,  the  Army  users  indicated  a  need  to  be  able  to  apply  HARDMAN  as  an 
assist  to  the  ASARC  or  program  management  budget  decisions.  Paramount  to 
making  program  direction,  budget  priority,  and  type  manpower  decisions  is 
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understanding  both  the  kinds  and  quality  of  data  employed  in  the  analysis. 
Consequently,  a  simple  data  rating  matrix  is  suggested  as  a  means  of  both 
summarizing  the  net  credibility  of  the  projections  in  the  executive  summary  of 
HARDMAN  applications  as  well  as  aiding  the  construction  of  the  consolidated 
data  base  in  Step  1  of  the  methodology.  Table  7-4  is  provided  as  an  example. 

Table  7-4  basically  applies  to  the  proposed  design.  The  listings  of 
both  the  weighting  criteria  and  data  source  quality  were  based  on  the  joint 
experience  of  the  Navy  and  contractor  HARDMAN  analysts.  The  table  has 
considerable  flexibility  in  that  the  analyst  can  fill  it  out  solely  for  the 
proposed  system  if  the  predecessor  reference  or  baseline  system  data  are  good 
or  evaluate  both  the  reference  and  proposed  systems  separately  if  each  con¬ 
tains  suspect  information.  Similarly,  the  data  evaluation  can  be  done  on  a 
component-by-component  basis.  In  all  these  schemes  the  analyst  merely  has  to 
indicate  the  availability,  maturity,  and  type  of  data  employed  that  auto¬ 
matically  places  the  outcomes  in  a  low,  marginal,  or  good  credibility 
quadrant.  Once  the  data  source(s)  has  been  located,  the  remaining  table 
elements  can  be  answered.  Probably  the  final  credibility  assessment  will  span 
two  quadrants  (i.e.,  low  to  marginal  or  perhaps  marginal  to  good).  Ultimately, 
a  summary  table  such  as  Table  7-4  could  provide  an  overall  assessment  of  the 
data  and  projections  to  the  decision  makers  so  that  MPT  planning  and  budgeting 
can  always  be  supported  by  the  more  reliable  projection  throughout  the  system 
development  phases. 

Processes  for  changing  or  manipulating  data  were  also  recommended  as 
part  of  the  suggested  improvements.  Although  not  as  complicated  as  the 
data-rating  scheme,  a  simple  delineation  of  helpful  hints  for  when  to  alter 
data  inputs  (along  with  supporting  examples)  is  necessary  to  help  clarify  the 
data  management  process  in  Step  1  and  improve  the  audit  trail.  Based  on  the 
findings  of  the  previously  described  audits,  the  basic  guidelines  for  handling 
data  appear  to  be  as  follows: 

(1)  When  building  the  consolidated  data  base,  first  check  the  proposed 
system  operator  and  maintenance  data  for  completeness;  if  data  are 
missing  for  any  components,  select  appropriate  component  data  from 
a  similar  existing  system,  perturb  the  data  per  instructions 
provided  in  Step  2  of  the  methodology,  and  complete  the  proposed 
system  data  set  (see  the  S1NCGARS  HARDMAN  results  for  an  example) 
(28). 

(2)  All  proposed  system  data  inputs  should  be  filtered  by  a  group  of 
experts  to  establish  confidence  that  the  data  are  reasonable.  For 
example,  basic  maintenance  repair  times  should  be  checked  to 
determine  if  sufficient  time  is  allowed  for  obtaining  test 
hardware  and  tools  by  comparing  them  against  similar  tasks  on 
predecessor  components.  If  erroneous  failure  rate,  indirect 
productivity,  or  induced  failure  data  are  encountered,  replace  the 
data  with  comparable  historical  values  (best  choice)  or  select 
appropriate  values  from  the  original  minimum  performance  criteria 
stipulated  in  the  request  for  proposal,  RFP  (next  best  choice) 

(see  the  RPV  HARDMAN  results  for  an  example)  (29). 
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(3)  Where  necessary,  it  sometimes  saves  time  to  use  more  recent 
historical  data  in  the  data  base  because  it  is  easier  to  collect. 
For  example,  if  the  more  recent  data  are  component  test  data,  then 
a  sample  sensitivity  study  should  be  conducted  to  determine 
whether  the  less  mature  (but  easier  to  obtain)  test  data  produces 
significantly  different  projections  than  the  older,  more  mature 
(but  harder  to  gather)  historical  data.  If  not,  then  replace  the 
historical  component  data  with  the  comparable  test  data  where 
appropriate  (see  the  RPV  HARDMAN  results  for  an  example)  (29). 

(4)  Occasionally,  it  may  be  difficult  to  find  a  predecessor  duplicate 
of  a  new  proposed  component.  In  this  situation  a  larger  or 
smaller  scale  component  with  similar  design  characteristics  may  be 
used  in  the  consolidated  data  base  as  a  last  resort.  However,  it 
is  extremely  important  to  rate  the  credibility  of  these  data  in 
terms  of  both  operational  similarities  and  environment  (see  RPV 
HARDMAN  results  for  an  example)  (29). 


b .  Suggested  Corrective  Actions  for  Perturbing  Reference  System 
to  Model  New  Proposed  Design.  A  framework  for  perturbing  predecessor  data  to 
fill  gaps  in  the  proposed  system  data  base  was  summarized  earlier  in  the 
audits.  To  provide  the  decision  makers  with  a  better  means  of  evaluating  the 
credibility  of  contractors'  projections,  it  is  necessary  to  compare  both  the 
total  reference  and  proposed  designs  on  an  equivalent  technology  basis.  In 
support  of  the  "evolutionary"  technology  growth  concept,  several  studies  have 
been  done  which  suggest  that  industry  is  often  too  optimistic  about  new 
technology  performance  (42).  Regardless  of  the  fact  that  much  of  the  overly 
inflated  performance  is  sometimes  derived  from  inadequate  testing,  it  seems 
that  a  20%  improvement  in  reliability  or  performance  can  reasonably  be 
expected  (41,42).  With  respect  to  Step  2  of  the  methodology,  a  simple 
top-level  adjustment  of  the  predecessor  reliability  and  maintenance  data  might 
be  structured  according  to  the  following  guidelines: 

(1)  If  the  design  difference  indicates  a  positive  improvement  and  the 
operating  environment  for  the  new  component  is  not  as  harsh  as  the 
predecessor  component,  then  employ  a  20%  increase  in  reliability 
with  a  "less  than"  or  "maximum"  label  on  the  manpower  projection 
(e.g.,  <  5,000  operators  required). 

(2)  If  the  design  difference  indicates  a  positive  improvement  and  the 
operating  environments  are  essentially  the  same,  increase  the 
component  reliability  by  20%. 

(3)  If  the  design  difference  is  insignificant,  or  the  positive  design 
improvement  is  offset  by  a  harsher  operating  environment,  then 
leave  the  reliability  and  maintenance  data  the  same. 


(4)  If  the  operating  environment  is  considerably  harsher  than  the 

predecessor  system  and  the  design  and  functional  differences  are 
insignificant,  then  do  not  perturb  the  predecessor  data,  and 
indicate  with  good  confidence  that  the  resulting  manpower, 
projections  represent  a  minimum  (e.g.,  a  minimum  of  5000  operators 
are  required;  or,  the  operator  requirement  is  >5000  people). 

With  these  indications  on  the  manpower  projection  for  the  updated 
predecessor  or  baseline  design,  one  can  start  to  identify  whether  the 
contractor  projection  for  new  technology  performance  is  reasonable  because  the 
projections  will  either  converge  or  diverge.  Along  with  the  earlier  credi¬ 
bility  ratings  on  the  various  kinds  of  data,  the  decision  makers  have  a  better 
means  of  making  budget  decisions  in  support  of  the  revised  predecessor  data  or 
the  contractor  projection. 


c .  Suggested  Corrective  Actions  for  Constructing  the  Reference 
System.  During  the  audits  it  was  discovered  that  the  HCM  designers  used  a 
more  sophisticated  way  of  selecting  predecessor  components  to  build  the 
reference  system  than  explained  in  the  methodology.  The  following  additional 
steps  are  suggested  to  help  clarify  and  better  structure  the  Step  1  reference 
system  construction  process: 

(1)  Include  a  complete  listing  of  documents  that  help  select 
predecessor  components  similar  to  the  proposed  system  (e.g.,  U.S. 
Army  Field  Manual  24-24,  Washington,  D.C.,  20  May  1977;  Department 
of  Army  (DOA),  Aviation  Electronics  Configuration  Directory, 
December  1970). 

(2)  Include  a  listing  of  typical  Army  and  industry  technical  contacts 
(by  organization  and  division)  in  various  component  classifications 
(e.g.,  radios,  surveillance  sensing,  etc.)  that  can  confirm 
similarities  or  differences  between  the  reference  and  proposed 
systems . 

(3)  Employ  two  reference  systems  if  two  proposed  systems,  although 
functionally  similar,  employ  completely  different  technologies 
(e.g.,  a  Howitzer  versus  a  rocket  launcher). 


d .  Suggested  Corrective  Actions  for  Functional-Requirements  Level  of 

Indenture .  Similar  to  the  preceding  reference  system  suggestions,  the  audits 
revealed  that  the  HCM  designers  employed  a  decision  process  which,  although 
simple,  provided  more  functional  and  task  requirements  guidance  than  the 
published  methodology.  It  is  suggested  that  the  following  steps  be  included 
in  Step  1  of  the  methodology  to  better  define  the  level  of  indenture  of  the 
functional  requirements: 

(1)  Go  to  a  level  in  which  the  associated  component  tasks  stop 

branching  off  (i.e.,  new  tasks  requiring  different  people  cannot 
be  identified,  such  as  a  mechanic  who  can  repair  both  the  truck 
engine  and  the  chassis). 
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(2)  Go  to  a  level  in  which  no  further  technology  differences  can  be 
delineated  between  components. 

(3)  Go  to  the  same  level  as  available  sample  predecessor  component 
maintenance  data  in  the  consolidated  data  base. 


e.  Suggested  Corrective  Action  for  MOS  and  Skill  Selections. 

The  audits  discovered  that  the  cookbook  approach  outlined  in  the  methodology 
was  not  complete.  The  additional  step  not  included  in  the  methodology,  which 
ensures  the  proper  matchup  between  tasks  and  skills,  is  the  use  of  experts 
from  various  proponent  schools.  Therefore,  it  is  suggested  that  the 
methodology  include  a  list  of  Army  proponent  school  contacts  (for  all  types  of 
component  groups  and  technologies)  that  can  confirm  or  alter  the  task  and 
skill  selections  as  necessary. 


f .  Suggested  Corrective  Action  for  Potential  End-Point  Manpower 
Errors .  The  solution  to  the  end-point  manpower  shortage  problem  was  outlined 
earlier  in  this  section.  The  basic  solution  revolved  around  shortening  the 
intervals  between  manpower  projections,  particularly  when  considering  manpower 
demands  of  emerging  systems  during  the  early  acquisition  or  later  life  cycle 
phases.  Figure  7-1  graphically  displays  the  solution  by  showing  MACR1T  pro¬ 
jections  at  one-year  increments  during  periods  of  rapid  changes  in  deployment 
and  attrition,  and  the  typical  long  term  MACRIT  rectangular  projection  for  the 
mid-cycle,  steady-state  period. 

g.  Suggested  Corrective  Action  for  Structured  Tradeoff 
Process .  To  instill  confidence  in  the  reader  that  the  methodology  encompasses 
all  possible  tradeoffs,  a  simple  table  could  be  constructed  as  shown  in 
Table  7-5.  Table  7-5  could  be  placed  in  the  executive  summary  as  a  means  of 
displaying  (1)  the  range  of  potential  tradeoffs  that  the  methodology  could 
provide  and  (2)  the  actual  tradeoffs  conducted  for  a  particular  application 
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Figure  7-1.  Revised  MACRIT  Manpower  Projection 
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as  constrained  by  time  or  the  sponsoring  agency.  The  actual  tradeoffs 
explored  would  be  indicated  by  placing  X’s  in  the  appropriate  slots  on  the 
matrix.  It  should  be  noted  that  Table  7-5  is  also  useful  if  it  is  desired  to 
display  a  clean  summary  of  the  various  parameter  sensitivities.  Typically, 
the  sensitivities  would  be  indicated  by  a  relative  percentage  change  (e.g.,  a 
IX  decrease  in  reliability  results  in  a  10Z  increase  in  manpower).  These  more 
detailed  cost  sensitivity  relationships  could  be  summarized  by  placing  the 
percentage  "deltas"  in  the  appropriate  matrix  slot.  For  example,  continuing 
with  the  reliability  illustration,  one  could  read  horizontally  across  the 
matrix  and  immediately  see  the  net  impact  on  the  other  LCC  variables.  This 
more  detailed  cost-sensitivity  matrix  could  be  used  to  cite  pivotal  cost  and 
tradeoff  benefits  and  jointly  close  out  Step  6,  tradeoff  considerations,  of 
the  methodology. 


h.  Suggested  Corrective  Action  for  Summarizing  Study 
Weaknesses .  Although  listing  study  weaknesses  is  a  simple  procedure,  it  is 
extremely  important  information  for  decision  makers.  In  conjunction  with  a 
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systematic  listing  of  weaknesses,  it  is  also  important  to  suggest  possible 
resolutions.  For  example,  it  is  perfectly  feasible  that  both  the  reference 
and  contractor  manpower  projections  could  have  "low-to-marginal"  credibility 
ratings  assigned  because  of  poor  or  insufficient  data.  Consequently,  -the  data 
problems  would  be  listed  as  weaknesses  and  a  suggested  resolution  might  be  to 
fund  test  programs  for  the  major  components  that  are  driving  the  projection 
credibility  down.  A  simple,  dual-heading  table  consisting  of  "Major  Study 
Problem  Areas"  and  "Suggested  Resolution  Actions"  could  be  placed  in  the 
executive  summary  to  assist  the  decision  makers.  This  adjustment  would  be 
fairly  minor  because  the  HCM  designers  already  address  problem  areas  in  some 
of  their  applications  (see  the  HARDMAN  SINCGARS  application)  (28). 


i .  Suggested  Corrective  Action  for  Projecting  Manpower  Out  to 
System  Deployment.  Examples  of  manpower  regression  models  were  provided 
earlier  in  Section  V.  The  key  problem  identified  with  these  models  was  the 
lack  of  any  indication  of  uncertainty  in  the  manpower  availability 
projections.  Nevertheless,  it  is  extremely  important  to  compare  the  projected 
manpower  requirement  against  manpower  available  starting  at  the  point  of 
system  deployment.  A  simple  initial  approach  to  this  dilemma  might  be  to 
employ  a  linear  regression  technique,  such  as  Barnes’  Soldier  90,  and  allow  an 
end  point  spread  representative  of  only  75  or  80%  confidence  (i.e.,  select  a 
confidence  interval  low  enough  to  prevent  too  large  a  spread  at  the  projected 
system  deployment  time)  (A3).  The  final  projection  may  appear  as  shown  in 
Figure  7-2,  where  the  shaded  area  might  represent  the  zone  of  confidence  in 
which  75  or  80%  of  the  observed  data  points  center  around  the  least  squares 
projection  (43). 


If  the  projected  manpower  requirement  resulted  in  the  situation  depicted 
in  Figure  7-2,  then  obviously  there  is  potential  for  a  critical  manpower 
shortage.  However,  if  the  projected  requirement  fell  within  the  shaded  area, 
then  for  purposes  of  early  planning,  it  would  appear  that  there 


Figure  7-2.  Use  of  Confidence  Intervals  for  Manpower  Projections 


was  no  immediate  problem.  The  advantage  of  this  approach  is  that  the 
projection  can  be  made  with  greater  confidence  as  the  deployment  date  is 
approached  and  more  data  points  on  manpower  availability  are  obtained. 
Probably  the  actual  confidence  in  the  projection  would  not  have  to  be  any 
better  than  one  minus  the  allowable  error  on  the  manpower  projection  (i.e., 
roughly  0.8,  as  determined  in  Section  Vl). 


j .  Suggested  Corrective  Action  for  Improving  LCC  Tradeoff 
Analysis.  As  shown  earlier  in  Section  V,  some  life-cycle  cost  elements  are 
already  included  in  the  HCM.  One  of  many  typical  life-cycle  cost  models  was 
also  briefly  summarized  (i.e.,  HLCCM).  It  is  merely  suggested  here  that  other 
life-cycle  cost  models  be  reviewed  and  the  remaining  missing  cost  parameters 
be  incorporated  in  the  methodology  to  form  a  complete  cost  and  impacts 
assessment . 


k.  Suggested  Corrective  Actions  to  Firm  Up  Perturbation, 

Induced  Failure,  and  Indirect  Productivity  Value  Ranges.  The  20%  upper  bound 
on  the  expected  performance  and  reliability  improvement  of  new  technology  was 
discussed  earlier  in  this  section.  This  expected  improvement,  however,  is 
only  one  part  of  the  complete  picture.  The  previous  discussion  also  pointed 
out  the  possibility  that  a  different  operating  scenario  or  harsher  environment 
could  drive  the  reliability  of  a  new  design  below  that  of  its  predecessor.  At 
a  top  level  it  was  merely  suggested  that  the  final  manpower  projection  be 
stated  as  "greater  than  X  number  of  people."  The  next  step  would  be  to  try  to 
establish  an  approximate  estimate  of  the  potential  drop  in  reliability.  A 
threefold  approach  is  suggested  to  obtain  this  estimate: 

(1)  Studies  such  as  the  earlier  referenced  Dobbins  paper  suggest 
roughly  an  order  of  magnitude  difference  between  contractor 
estimates  of  reliability  and  actual  field  operation  (42).  It  may 
be  enlightening  first  to  determine  the  components  involved  in  the 
above  Dobbins  reliability  tests,  choose  equivalent  predecessor 
components,  compare  the  operating  environments  in  terms  of 
relative  harshness,  and  calculate  the  incremental  difference 
between  the  respective  reliabilities  to  get  an  idea  of  whether  new 
technology  (operating  in  a  different  environment  and  adjusted  to 
its  realistic  performance  level  according  to  Dobbins)  experiences 

a  reduction  in  reliability  below  its  predecessor. 

(2)  As  a  next  step  it  may  be  useful  to  conduct  a  survey  of  program 
managers  in  all  four  services  to  gather  examples  (and  maintenance 
data)  of  systems  and  components  where  the  reliabilities  fell  far 
short  of  their  original  goals.  Again,  establish  the  equivalent 
predecessors  that  were  replaced  by  those  new  designs,  compare  the 
harshness  of  the  operating  environments,  and  calculate  the 
incremental  difference  in  reliability  between  the  two.  Several 
different  kinds  of  components  should  be  examined  to  determine 
which  technologies  are  more  sensitive  than  others  (i.e., 
computers,  propulsion  components,  firing  devices,  etc.). 
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(3)  Finally,  establish  a  group  of  experts  from  various  technical 

fields  in  DOD  to  adjust  the  estimates  considering  that  research 
and  development  programs  provide  engineering  fixes  that  allow  some 
of  the  reliability  to  be  regained  before  a  fairly  steady-state 
reliability  level  is  reached. 

The  induced  failure  and  indirect  productivity  variables  can  effect  the 
outcome  of  the  manpower  projection  even  though  their  ranges  are  not  as  broad 
as  the  perturbation  value.  As  a  first  step  it  is  suggested  that  a  sensitivity 
analysis  be  conducted,  using  completed  HARDMAN  applications,  by  simply 
calculating  the  spread  in  manpower  for  the  complete  ranges  of  both  variables. 
If  the  spread  does  not  aggravate  the  sensitivity  of  the  manpower  variable, 
then  a  single,  perhaps  mid-point,  value  should  be  selected  to  remove  any 
confusion  in  picking  a  value  from  the  possible  ranges.  If  the  spread  exceeds 
the  allowable  error,  then  more  detailed  breakdowns  using  industry  and  services 
industrial  engineering  studies  (for  indirect  productivity  factors)  or  the  Navy 
200,000  Hour  Test  Study  (for  human  induced  failure  rates)  will  have  to  be  used 
(41). 


2.  Training  of  HARDMAN  Analysts 

One  of  the  major  results  of  the  audits  and  reliability  analyses 
was  that  the  methodology  appeared  usable  by  only  a  select  group  of 
individuals.  Comments  made  during  the  debriefing  following  the  internal 
validity  test  universally  suggested  that  no  one  individual  could  exercise  the 
methodology  reliably;  a  team  effort  was  absolutely  required.  It  was 
discovered  that  the  net  expertise  required  was  a  combination  of  the  following: 

(1)  Sufficient  military  experience  to  provide  both  an  understanding  of 
where  to  obtain  training  and  technical  information,  as  well  as  the 
structure  of  the  military  logistics  support,  personnel  and 
training  environments. 

(2)  Technical  capability  in  at  least  one  of  a  wide  range  of  technical 
fields  encompassing  mathematics  (particularly  modeling), 
behavioral  sciences,  aerospace  engineering,  mechanical 
engineering,  electrical  engineering,  and  chemical  engineering. 

(3)  Experience  in  computer  information  and  data  management  systems. 

(4)  Experience  with  DOD  personnel  assignment  and  tracking  processes. 

Again,  based  on  comments  made  by  replication  analysts  during  the 
debriefing  and  discrepancies  review  (41),  it  seems  that  the  most  suitable 
combination  of  the  above  abilities  was  either  items  (1),  (2),  and  (3),  or 
items  (1),  (2),  and  (4).  Note  that  the  joint  military  and  technical 
experience  recur  in  both  combinations.  The  personnel  experience  could  be  an 
extension  of  the  military  experience;  and,  similarly,  the  computer  information 
system  experience  could  greatly  enhance  the  technical  experience.  All  of 
these  attributes  exist  to  varying  degrees  in  the  individuals  employed  by  the 
HCM  contractor  who  execute  the  HARDMAN  methodology  (31).  While  some  of  the 
first-year  methodology  design  recommendations  are  being  explored,  the  Army 
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might  consider  using  these  qualifications  to  select  and  staff  its  HARDMAN 
analyst  team(s).  In  preparation  for  possibly  conducting  in-house  applications 
of  the  methodology  or  managing  and  auditing  contractor  applications,  it  is 
extremely  important  that  the  Army  firm  up  the  various  judgment  areas  per  the 
preceding  recommendations.  This  effort  will  also  provide  for  a  clearer 
training  foundation.  Drawing  on  both  the  results  of  the  preceding  evaluation 
and  debriefing  comments,  the  following  training  suggestions  are  provided: 

(1)  The  original  objective  of  the  HARDMAN  training  session  (in 
preparation  for  t'  e  internal  validity  test)  was  to  concentrate  on 
developing  skills  for  the  judgmental  portions  of  the  methodology. 
This  objective  should  remain  the  major  training  thrust  because  the 
algorithms  and  number  manipulation  are  fairly  straightforward. 
Comments  made  on  the  debriefing  questionnaire  suggest  that  the 
first  training  session  concentrated  too  heavily  on  the  numbers  and 
data  manipulation. 

(2)  The  handouts  (particularly  the  LSA  data  and  examples)  should  be 
designed  and  their  presentation  associated  more  clearly  with  each 
step  and  judgment  in  the  process.  The  emphasis  should  be  placed 
on  defining  the  judgment  process  and,  where  calculations  are 
required,  concentrate  on  only  a  few  select  cases  that  demonstrate 
the  possible  range  of  judgments  that  may  be  encountered.  Note 
that  the  suggested  corrective  actions  made  earlier  in  this  section 
could  help  refine  the  judgment  process  and  that  some  of  the 
peculiar  data  management,  reference  system  construction,  and 
credibility  problems  discussed  earlier  as  part  of  the  SINCGARS, 
RPV,  and  DSWS  audits  could  be  employed  as  example  problems. 

(3)  Break  down  the  process  by  areas  of  expertise  and  provide  different 
focuses  for  the  backgrounds  of  various  individuals  (i.e.,  teach 
the  team  approach  and  define  participant  responsibilities). 

(4)  Train  for  the  complete  six-step  methodology  (e.g.,  as  part  of  the 
training  program  it  may  be  useful  to  have  the  Army  analysts  work 
as  apprentices  at  the  HCM  contractor's  facility  as  a  means  of 
obtaining  experience). 

These  recommendations  should  not  be  construed  as  only  applying  to 
HARDMAN  analysts.  Even  if  the  Army  only  chooses  to  monitor  contractors 
performing  applications  in  response  to  a  Data  Item  Description  (DID)  in  a 
contract,  it  is  still  imperative  that  contract  monitors  have  the  recommended 
training  to  ensure  quality  control  of  the  outcomes  and  answer  ASARC  or  Program 
Office  audit  requests. 


3.  Suggestions  for  Follow-on  Research 

There  were  four  far-term  areas  identified  earlier  in  this 
section.  These  areas  revolved  around  consideration  of  emerging  systems 
(horizontal  analysis),  quantitative  differentiation  between  critical  and 
non-critical  tasks,  expansion  of  the  demographic  data  base,  and  consideration 
of  major  socioeconomic  fluctuations.  Except  for  horizontal  analysis,  the 
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remaining  areas  are  probably  not  critical  to  early  Phase  0  and  Phase  1 
top-level  budget  planning.  As  a  first  step,  it  is  suggested  that  the  Army 
initiate  appropriate  directives  to  require  early  planning  of  "most  likely" 
candidates  to  be  approved  as  emerging  systems,  inclusive  of  early  operational 
and  projected  logistics  support  data.  It  is  recognized  that  this  is  a  large 
and  perhaps  sketchy  task.  However,  if  the  Army  wishes  to  meet  its  user  needs 
and  consider  major  drains  of  other  systems  on  the  personnel  pool,  it  must 
start  the  mechanism  to  gather  the  needed  information  as  soon  as  possible. 

The  three  remaining  areas  are  considered  far-term  because  they  represent 
large  data-gathering  activities.  For  example,  the  quantitative  critical  and 
non-critical  task  differentiation  effort  might  be  accomplished  by  (1)  first 
establishing  criteria  for  critical  and  non-critical  tasks  (e.g.,  life  or 
mission  endangering,  task  difficulty);  (2)  selecting  an  array  of  systems 
representative  of  obvious  task  extremes  for  operator  or  maintainer  performance 
(e.g.,  aircraft  pilot  versus  an  RPV  operator);  (3)  calculating  the  ratios  of 
instructor  hours  for  the  extremes;  and  (4)  developing  a  simple  rule  of  thumb 
to  increase  instructor  contact  hours  by  "X"  percent  in  Step  3  of  the 
methodology  for  those  tasks  judged  critical.  The  time-consuming  aspect  of  an 
effort  such  as  this  is  the  task  sorting  and  rating  study  associated  with  item 
(1)  followed  by  the  gathering  of  system  examples  and  instructor  task  hours 
associated  with  items  (2)  and  (3).  It  is  for  these  reasons  that  the  "obvious 
task  extreme"  qualifier  was  suggested  as  a  starting  point. 

The  completion  of  the  demographic  data  base  is  a  massive  effort  because 
it  not  only  involves  the  gathering  of  data  on  age,  sex,  education,  etc.,  but 
also  requires  expansion  of  the  decision  network  in  Step  2  of  the  HCM  (i.e., 
skill  and  MOS  assignments).  It  may  be  worthwhile  for  the  near  and 
intermediate  terms  not  only  to  require  manufacturers  contractually  to  adher 
strictly  to  accepted  human  engineering  guidelines  but  also  to  establish  a 
design  auditing  mechanism  to  ensure  strict  compliance  with  sound  human 
engineering  standards. 

The  last  area,  socioeconomic  impacts  on  manpower,  should  be  pursued  in 
the  manner  expressed  by  Enke  earlier  in  the  document.  In  the  interim,  a 
subjective  weighting  mechanism  might  be  employed  to  indicate  the  general 
direction  of  manpower  trends  (up  or  down),  based  on  expert  experience  with  the 
various  variables  (such  as  the  economy).  Referring  back  to  the  earlier 
recommendation  on  the  use  of  confidence  intervals  around  the  long-term 
manpower  availability  projection,  the  final  manpower  projection  might  simply 
indicate  that  the  most  likely  availability  population  would  be  either  above  or 
below  the  least  squares  linear  projection,  depending  on  the  historical  effect 
of  the  socioeconomic  variables. 

The  final  far-term  research  area  refers  to  one  of  the  original 
constraints  imposed  on  this  evaluation.  Due  to  time  and  budget  constraints, 
only  a  qualitative  examination  of  methodology  accuracy  was  feasible.  It  is 
strongly  suggested  that  a  feedback  loop  be  constructed  so  that  actual  field 
data  on  completed  applications  are  gathered  as  the  new  systems  are  deployed. 

If  the  actual  system  MPT  performance  falls  outside  the  allowable  error 
discussed  in  Section  VI,  then  appropriate  steps  should  be  taken  to  backtrack 
through  the  studies  and  strengthen  those  weak  areas  in  the  methodology. 


In  conclusion,  several  different  validity  examinations  were  structured 
to  evaluate  the  HCM.  Table  7-1  indicates  that  the  technique  was  generally 
sound  and  basically  integrated  all  aspects  of  the  MPT  problem  well.  Although 
some  weaknesses  were  discovered,  solutions  (or  solution  structures)  were 
apparent  in  most  cases  and  appeared  attainable  within  a  reasonable  time 
period.  If  the  Army  chooses  to  pursue  the  proposed  recommendations,  several 
benefits  might  be  realized: 

(1)  The  Army  will  move  closer  to  developing  its  own  in-house  HARDMAN 
capability  (if  it  elects  to  do  so). 

(2)  The  Army  will  improve  the  audit  trail  on  MPT  projections. 

(3)  The  Army  will  have  a  sound  tool  for  producing  MPT  requirements, 
weighing  the  credibility  of  projections,  and  making  better 
personnel  recruitment  and  budget  decisions  in  the  manpower  support 
arena. 
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APPENDIX  A 

INTERVIEW  QUESTIONS  FOR  USER  REQUIREMENTS 


I 

I 


USER  REQUIREMENTS 
QUESTIONNAIRE 


1.  To  what  organizational  division  of  the  Army  do  you  belong? 
VERBATIM 


2.  What  ia  the  primary  charter  of  your  organization? 
VERBATIM 


3.  What  are  your  specific  duties  within  the  organization? 
RESPONSIBILITY :  (Areas  of  Concern,  Deliverables,  etc.) 


4.  What  are  your  key  needs  in  making  or  using  manpower  personnel  and  training 
projections  (i.e.,  what  do  you  need  to  have  to  do  this?) 

INPUTS:  Type  of  data.  Level  of  Detail 


INTERFACE  CONSIDERATIONS 

Other  Organizational  Actors  (input/output) 


Timing 


5.  What  are  the  key  drivers  in  needing  such  MPT  projections,  e.g.,  cost, 
planning,  safety,  performance,  readiness/force  modernization,  etc? 
SPECIFY  AND  EXPLAIN,  if  possible 


6.  Which  ARMY  MPT  regulations/documents  pertain  to  your  organization's 
involvement  in  the  M.P.T.  process? 


7.  What  technique  is  presently  employed  for  getting  MPT  inputs  into  system 
acquisition  process? 


s 

8.  Who  has  responsbility  for  this  technique*  , 


9.  To  your  knowledge,  does  it  provide  good  results? 
SPECIFY 


10.  In  your  organization,  who  ia  (are)  the  most  likely  user(s)  of  a 
methodology  to  make  MPT  projections? 


11.  What  other  usera  of  the  the  methodology  could  you  identify  within  the  Army 
organizational  structure? 

12.  Who  in  your  organization  ia  (are)  the  moat  likely  uaer(a)  of  the  re8ul ts 
of  auch  a  methodology? 

13.  What  other  uaera  of  the  reaulta  of  auch  a  methodology  could  you  identify 
within  the  Array? 

14.  What  auggeationa  do  you  have  for  the  deaign  of  a  methodology  to  accompliah 
MPT  projectiona?  (OUTPUTS,  SCHEDULE,  PROCESS,  ETC.) 

15.  What  would  be  the  moat  uaeful  format(a)  for  the  reaulta? 

16.  At  what  point  in  the  acquisition  process  (i.e.  Phase  0,  I,  etc.)  do  you 
require  MPT  results  for  (a)  deaign  changes,  or  (b)  coat  control? 


HARDMAN  USER  REQUIREMENTS  SUMMARY  MATRIX  (DESIGN)  (Continued) 


HARDMAN  USER  REQUIREMENTS  SUMMARY  MATRIX  (OUTPUT) 


1  OPERATOR  (MAMTAMEA  TASK  DATA  ASSOCIATED  WITH  A 
SYSTEM  AMD  ITS  SUPPORT!  TEST  EQUIPMENT  DESCRIPTION 
AMO  FREQUENCIES) 


2  MAMTEMAMCC  MANHOURS  (MANPOWER  DfMAMO  IASEM  fl 
ACCIMATE  MUM  DAEA 


3  MOS.  SKILL  LEVEL  REQMREMENTS.  AOOfTIMAL  SOU 
MOCATORS  IASI  AMO  TOTAL  ROOMERS  OF  IMCU 


4  MAMTE  NANCE  CONCEPT.  MCUIOMO  MAMTE  NAME  LEVELS. 
MTTA.  MTBF 


6  COMPREHENSIVE  TRAM  IMG  CONCEPT 

■CUSSES  REQOMED  (UNfTfMDtVMNSTITOTIQRAL) 
-TRAMMC  TIME  FREQUENCY 
QUANTITATIVE  AMO  QUALITATIVE  TRAMMC  M CREASES 
TRAMMC  SUPPORT  (DEVICES  NSTMICTORS.  FAOLITKS) 


I-  PERSONNEL  AMUUUUTYfCORSTRAMTS 
PERSONNEL  PROJECTIONS.  TRENDS 


7  MPT  COSTS  ASSOOATIO  WITH  SYSTEM  OPERATION  AM 
SUPPORT 


I  TRADEOFFS  IETWCEN  HARDWARE  AM  PEOPLE 


•  ACCURATE  ISA  DATA  RASED  ON  COMPARISON  TO  NfPlACED 
SYSTEM 


ID  HISTORICAL  DATA  RASE  RJPOATE  CAPRiOfTY} 


II-  HPT  “DATA  SUCT  RMSMTATM  DP  BAM  MTT  BP 
BtPIBVN— T  LAB  EBWPPIB  MT.  SMPI  AS  A  BATTAUM. 
COMPANY.  ITCJ  MS* 


14-  TARBCT  AMDNBCf  NESONPTMN 


11  MOS  TUJBOBMOT’  BASR  ME:  CURRENT  MBS  STATBS: 


IS-  DfTAAEP  OPERATIOBAL  AM  BMABOAnMAL  MAO)  CMCfPT 
FOB  THE  SYSTEM  MCLUOMC  TBE  DATA 


ll-MATMC  ISA  BAZA  FROM  FKLOCO  SYSTEMS  TO  VfMPY 
M FORMATION  SYSTEM  AM  PERMIT  ANAlTStt  OF  SYSTEM 


20-  SBMPU  FORMATS.  MCLBMK  CHARTS.  TABUS.  OMPM 


21-  PEKOE  TASK  NUKOFNI  FOB  EACH  MBS  ABBBQATlO  WTTM 
A  SYSTEM 


22  HARDCOPY  AM  CRT  DM  PUT  CAPABILITY  PERMANENT 


21  EXECUTIVE  SUMMARY  REPORT  TO  ACCOMPANY  OUTPUTS 


24-  OUTPUT  SHOULD  BE  BAIR  NEEDED  M  LETTER  OF  ABRHMEBT 
(SEE  111 


METHODOLOGY  AUDIT 


Generic 


Steps  1  and  2 

1.  What  judgment  determines  the  level  of  indenture  necessary  in  determining 
the  functional  requirements? 

2.  How  does  one  determine  the  maturity  of  the  data  presented? 

How  does  on  ensure  selecting  the  most  mature  data  source(s)? 

3.  The  judgments  used  to  determine  manpower  drivers  in  the  design 
difference  index  are  not  clear  to  me: 

How  are  the  key  drivers  flagged? 

How  is  the  perturbation  value  assigned? 

When  and  where  is  the  perturbation  value  applied? 

4.  What  judgment  is  required  and  how  is  it  applied  to  select  the  lowest 
skill  level  MOS  for  any  maintenance  task  when  the  AR61 1-201  seems  to 
define  the  performance  of  each  MOS  quite  clearly? 

5.  What  judgments  temper  the  use  of  the  indirect  productivity  factor?  Is 
this  a  blanket  application  or  a  selected  application?  Define  why  and 
when. 

6.  What  judgments  allow  the  incorporation  of  two  tasks,  normally  performed 
by  two  separate  MOS's,  into  one  MOS? 

7.  What  judgments  are  used  in  creating  a  new  MOS? 

8.  When  and  where  are  the  perturbation  values  applied  to  the  reference 
system  to  achieve  a  baseline  system  with  which  to  compare  the  contractor- 
proposed  data? 

9.  Is  the  function  to  address  the  manpower  loading  a  rectangular  or  weible 
curve? 

10.  How  is  the  accuracy  of  the  reference  system  ensured  when  contractor- 
proposed  data  is  used  in  building  a  reference  system  component  for  which 
no  predecessor  data  is  available? 

11.  How  do  you  construct  a  reference  system  to  be  similar  to  the  proposed 
system  when  the  proposed  systems  vary  widely? 

12.  Does  the  HARDMAN  Methodology  take  into  account  factors  other  than  those 
addressed  by  the  contractor?  Are  the  DRC  reference  system  manpower 
requirements  more  credible  than  the  contractor-proposed  requirements? 


C-3 


METHODOLOGY  AUDIT 


Generic 


Is  Che  training  necessary  for  Che  instructors  taken  into  account? 
Training  for  the  instructor's  instructors? 

Are  the  costs  of  training  the  instructors  taken  into  account?  The  costs 
of  training  the  instructor's  instructors? 

What  judgments  are  used  in  identifying  and  constructing  training  courses 
for  any  new  ratings  developed? 

When  the  technical  factors  for  determining  the  baseline  skill  levels  are 
analyzed,  is  memory  decay  taken  into  account? 

Explain  the  two  levels  of  indenture  for  determining  training 
requirements;  the  Methodology  seems  to  address  only  the  first. 

What  determines  the  frequency  or  need  for  refresher  (retraining)  courses? 

How  does  the  TRRA  estimate  the  cost  of  devices  required  for  training? 

Are  the  costs  for  personnel  movement  between  training  schools  and 
training  periods  included  in  training  costs?  Are  these  large  costs? 

How  often  is  it  required  to  send  personnel  back  through  the  training 
schools?  Is  this  a  large  cost? 

How  are  the  perturbation  values  determined  in  steps  1  and  2  applied  to 
the  reference  system  training  courses? 


METHODOLOGY  AUDIT 


Generic 


Step  A 

1.  How  are  the  personnel  required  for  new  MOS's  accounted  for? 

2.  Where  is  the  flow  of  second  enlistment  personnel  taken  into  account? 

3.  How  is  a  personnel  flow  conducted  for  a  new  MOS  when  no  historical  data 
is  available? 

4.  Where  do  the  probabilities  to  calculate  the  flow  of  personnel  come 
from?  How  are  they  applied  to  the  MOS  populations  to  establish  the 
expected  values  of  availability? 

5.  Are  major  socioeconomic  fluctuations  considered? 

6.  Is  a  steady-state  population  within  the  Army  assumed? 

7.  How  are  the  perturbation  values  determined  in  steps  1  and  2  applied  to 
the  reference  system  personnel  requirements? 


Conclusions 

1.  Why  is  a  Life  Cycle  Costing  (LCC)  model  not  included  in  the  Methodology 

2.  When,  if  at  all,  is  a  wartime  scenario  addressed? 

3.  How  much  credibility  is  lent  to  the  results  of  the  analysis? 


METHODOLOGY  AUDIT 


DSWS/RPV-Specif ic 


Steps  1  and  2 
DSUS 

1.  What  justification  does  PACCAR  have  for  utilizing  an  "M-2-like"  chassis 
in  a  potential  overload  situation? 

2.  What  accounts  for  the  vast  differences  in  manpower  between  the  Norden 
M109E4  system  and  the  reference  and  other  proposed  systems? 

3.  Why  weren't  personnel  and  training  analyses  performed  on  the  PACCAR 
system? 

4.  The  Norden  M109E4  system  has  no  Ammunition  Resupply  Vehicle  (ARV).  The 
PACCAR  system  uses  only  one  ARV  per  two  Howitzers.  How  do  these  two 
systems  get  considerations  as  proposed  systems  if  the  functional 
requirements  were  developed  properly?  Do  you  consider  the  fact  that  the 
PACCAR  system  uses  one  ARV  per  two  Howtizers  a  potential  error  source? 

5.  Why  was  a  shipboard  gun  autoloader  chosen  for  the  reference  system? 

6.  For  the  following  subsystems,  what  judgments  were  used  in  determining 
the  design  differences?  What  judgments  were  used  in  assigning  a 
perturbation  value?  How  were  these  values  applied  to  the  reliability 
and  maintainability  data?  How  did  they  affect  the  manpower  projections 
for  the  reference  system? 

a.  Autoloader 

b.  Fire  Control  Processor 

c.  Internal  Navigation  System 

d.  Auxiliary  Power  Unit 

e.  Suspension:  1.  Howitzer 

2.  ARV  (where  applicable) 

f.  Howitzer  Engine 

g.  Suspension/ Chass is  Combination 

7.  In  building  the  reference  system,  what  criterion  were  used  to  determine 
the  maturity  of  the  data? 

8.  How  does  the  use  of  a  trailer  with  the  FMC  system  affect  the  reliability 
and  maintainability  of  that  system?  How  does  the  use  of  a  trailer 
affect  the  raunitions-loading  capabilities  of  that  system? 

9.  What  accounts  for  the  vast  differences  in  manpower  projections  between 
the  reference  and  proposed  systems  in  the  following  MOS  categories: 

31V,  31E,  35E,  630. 

10.  What  accounts  for  the  extremely  low  number  of  the  following  MOS’s  in  the 
Norden  M109E4  system:  31V,  31E. 
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METHODOLOGY  AUDIT 


DSWS/RPV-Specif ic  (cont'd.) 


Step*  1  and  2 

DSWS  (cont'd.) 

11.  Why  was  an  airborne  unit  selected  as  the  reference  navigation  system? 

12.  What  justification  exists  for  Norden  and  FMC  systems  not  using  an 
Auxiliary  Power  Unit? 

13.  How  was  the  M-l  tank  Auxiliary  Power  Unit  data  altered  to  take  into 
account  its  lack  of  predicted  performance? 


RPV 

1.  Track  the  source  of  the  Mean-Time-To-Repair  (MTTR)  values  of  the 
reference  system  for  the  following  subsystems: 

a.  Data  link  terminal  (airborne  unit) 

b.  Airframe  assembly 

c.  Launcher 

2.  Identify  the  reference  equipment  and  selection  rationale  for  the 
following  subsystems: 

a.  Central  Processor  Unit  (CPU) 

b.  RPV  control  and  display  panel 

C.  Airframe  assembly 

d.  AV  engine 

e.  Launcher 

3.  Track  the  Contractor-Furnished  Equipment  data  from  the  baseline  system 
equipment  that  was  used  in  the  reference  system  for  the  following 
subsystems : 

a.  Launcher 

b.  CPU 

c.  Airborne  unit  data  terminal 

4.  Identify  the  subsystems  of  the  baseline  system  that  did  not  meet  the 
functional  requirements. 
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Identify  the  baseline  equipment  that  used  altered  contractor  data. 
Explain  the  rationale  for  altering  this  data. 


METHODOLOGY  AUDIT 


DSWS/RPV-Specif ic  (cont'd.) 


Steps  1  and  2 


RPV  (cont’d.) 

6.  Identify  and  provide  the  rationale  for  the  perturbation  values  (PV) 
assigned  to  the  following  subsystems: 

a.  Airborne  unit  data  terminal 

b.  CPU 

c.  AV  airframe 

d.  AV  engine 

If  no  PV  was  used  in  the  above  subsystems,  identify  where  the  PV  was 
used. 

7.  Identify  baseline  equipment  where  no  reliability  and  maintainability 
(R&M)  data  was  available. 

8.  Track  MOS/Skill  level  assignments  for  the  following  subsystems: 

a.  Airborne  unit  data  terminal 

b.  CPU 

c.  Airframe  assembly 

d.  AV  engine 

9.  Page  48  of  Volume  1,  RPV  states  that  the  assumption  is  made  that  the  MAC 
charts  exclude  the  make-ready/put-away  times,  a  value  which  would 
normally  be  adjusted  by  the  Indirect  Productivity  Factor.  In  the 
SINCGARS  replication,  the  instructions  stated  that  the  make-ready/ 
put-away  times  were  included  in  the  MAC  chart  data.  Explain  this 
discrepancy. 

10.  Explain  how  full-sized  aircraft  data  was  altered  in  order  to  meet 
miniaturized  aircraft  data  projections. 

11.  Page  75  of  Volume  1,  RPV  makes  reference  to  certain  modeling  equations. 
Explain  these. 

12.  What  is  the  basis  for  a  two-to-one  improvement  of  the  M939  truck  over 
the  M809? 


13. 


What  criterion  are  used  in  determining  the  number  of  systems  to  be 
fielded?  Are  these  criterion  adjusted  for  system  availability? 


METHODOLOGY  AUDIT 
DSWS/RPV-Specific  (cont'd.) 


Step  3 


DSWS 

1.  How  did  the  perturbation  values  assigned  to  components  of  the  following 
subsystems  of  the  reference  system  affect  the  training  resources 
requirements  analysis? 

a.  Autoloader 

b.  Fire  Control  Processor 

c.  Internal  Navigation  System 

d.  Auxiliary  Power  Unit 

e.  Suspension:  1.  Howitzer 

2.  ARV  (where  applicable) 

f.  Howitzer  Engine 

g.  Suspension/ Chassis  Combination 

2.  What  training  impacts  are  effected  by  the  use  of  a  trailer  in  the  FMC 
system? 

RPV:  For  Questions  1  and  2 

References:  RPV  Appendix  C  -  pp.  3,  A,  27,  30,  57 

Subsystems:  Air  Vehicle,  airborne  data  terminal,  remote  ground 
terminal 

1.  What  was  the  rationale  for  assigning  the  PV's?  What  are  the  values  that 
were  arrived  at? 

2.  What  is  the  rationale  for  choosing  the  respective  representative 
equipments  for  training  estimation?  What  would  the  procedure  be  if  a 
detailed  Training  Resources  Requirements  Analysis  (TRRA)  was  conducted? 

3.  Identify  the  nine  new  courses  in  the  reference  system;  the  seven 
modified  courses  in  the  baseline  system;  the  two  deleted  courses  in  the 
baseline  system. 

4.  Discuss  the  implications  of  the  fact  that,  in  many  cases.  Direct  Support 
(DS)  level  maintenance  was  not  specified  in  the  Logistics  Support 
Analysis  (LSA)  data. 

5.  Discuss  the  low  performance  that  may  result  from  high  skill  requirements 
and  low  paygrade  assignments  as  indicated  on  page  119,  RPV;  explain  why 
this  was  not  identified  in  the  recommendations;  how  was  this  addressed 
in  the  reference  system? 


METHODOLOGY  AUDIT 


DSWS/RPV-Specific  (cont'd.) 


Step  3  (cont'd.) 

RPV:  (cont’d.) 

6.  Explain,  by  example,  the  four  steps  in  determining  the  number  of 
instructors  for  one  course. 

7.  Show  an  example  of  a  Program  Of  Instruction  (POI)  used  for  obtaining 
training  man-days. 

8.  Since  the  HARDMAN  Methodology  does  not  address  Life  Cycle  Costing  (LCC) , 
and  only  addresses  a  potion  of  the  training  costs,  why  address  it  at 
all?  Why  wasn't  it  addressed  in  RPV? 

9.  In  general,  how  does  this  step  interrelate  with  the  M0S/AS1  selection  in 
step  1? 


Step  4 


DSWS 

1.  How  did  the  perturbation  values  assigned  to  components  of  the  following 
subsystems  of  the  reference  system  affect  the  personnel  requirements? 

a.  Autoloader 

b.  Fire  Control  Processor 

c.  Internal  Navigation  System 

d.  Auxiliary  Power  Unit 

e.  Suspension:  1.  Howitzer 

2.  ARV  (where  applicable) 

f.  Howitzer  Engine 

g.  Suspension/Chassis  Combination 

2.  What  personnel  requirements  are  affected  by  the  use  of  a  trailer  in  the 
FMC  system? 


RPV 

1.  Does  the  primary  output  of  this  step,  personnel  to  be  trained  per  year, 
represent  requirements  at  IOC,  5  yrs.,  10  yrs.,  .  .  .  ? 

2.  Discuss  the  sources  of  personnel  flow  rates.  Why  does  DRC  rely  on  Army 
data  and  not  develop  its  own  models,  or  use  the  algorithms  outlined  in 
the  HARDMAN  Methodology  handbook?  What  is  the  expected  error  in  the 
primary  output  due  to  inadequate  data  inputs? 

3.  Explain,  by  example,  personnel  flow  in  the  13T  MOS  structure. 

4.  Are  there  other  Army  data  sources  available,  such  as  TKADOC? 
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Step  5 


RPV 

1.  Page  154,  RPV  states  that  meaningful  results  can  be  obtained  in  the  case 
of  RPV  since  it  is  a  completely  new  system.  Does  this  imply  that 
meaningful  results  could  not  be  obtained  if  this  were  not  the  case? 

2.  Does  DRC  provide  inputs  or  actually  run  the  Army's  Personnel  Policy 
Project  Model  (P^M)? 

3.  The  P^M  Model  projects  personnel  suply  based  on  Army  internal 
variables;  what  of  external  variables  such  as  the  economy  or  politics? 

4.  Are  the  P^M  personnel  inputs  different  from  those  used  by  DRC  in  step 
4  of  the  Methodology? 

5.  Given  the  short  horizon,  one  to  three  years,  of  the  P-hl  Model,  is 
there  a  better  way  and  why? 

6.  How  do  authorizations  relate  to  the  TO&E? 


Step  6 

RPV 

1.  How  long  does  it  take  to  conduct  a  tradeoff  study? 

2.  Discuss,  by  example,  the  tradeoff  study  of  the  M939  truck  versus  the 
ML809  truck. 


Conclusion 


1.  What  conclusion  can  you  draw  from  the  chart  on  page  5,  RPV;  what 
conclusions  can  we  draw  on  how  credible  the  answers  are? 
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APPENDIX  D 

DEBRIEFING  QUESTIONNAIRE 


TRAINEE  DATA  COLLECTION  SHEET 


HARDMAN  PROJECT 


In  order  to  analyze  and  identify  possible  explanations  for  divergence  in 
trainee's  solutions  to  problems  worked  during  the  training  course,  we  need  to 
determine  the  following  background  data  on  each  person. 

1.  TRAINEE  I.D.  #  (NAME) 

2.  LEVEL  OF  EDUCATION 


NO.  YEARS  COMPLETED 

ADVANCED  DEGREES? _ 

CERTIFICATES?  _ 

COLLEGE  DEGREE  _ 

HIGH  SCHOOL  _ 

3.  TYPE  OF  EDUCATION 


ADVANCED  DEGREE  FIELD(S) 

COLLEGE  MAJOR  _ 

H.S.  EMPHASIS  _ 

COLLEGE  PREP  _ 

TECHNICAL  _ 

VOCATIONAL  _ 

OTHER 


A.  PRESENT  EMPLOYMENT 


TITLE: 


EMPLOYER: 


DUTIES: 


LENGTH  OF  TIME  (IF  ANY)  AWAY  FROM  PAST  EDUCATION  FIELD  OF  EMPHASIS 
(MAJOR,  FIELD,  ETC.) 

5.  PAST  EMPLOYMENT 


IN  REVERSE  CHRONOLOGICAL  ORDER,  PLEASE  LIST  RECENT  JOB  EXPERIENCE  (UP 
TO  5  YEARS) 
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DEBRIEFING  QUESTIONS 

1.  Did  you  complete  all  8  sections  of  the  replication  paekuge? 
If  NO,  state  which  portion  is  incomplete... 


K.m 
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2.  Were  the  recommended  time  frames  realistic? 

If  NO,  state  specific  packages  which  tobk  more/less  time  than  recommended. 


3.  Was  the  subject  material  or  methodology  of  this  replication  process  directly 
or  indirectly  related  to  your  present  job? 

If  YES,  state  how  (briefly) 


V.-/ 

& 
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Ware  there  portions  of  the  replication  that  yon  felt  were  particularly 
difficult?  (Beginning  with  package  1,  please  give  any  significant  notations 
you  made  about  the  package  while  working  on  the  replication.  Please  omit 
problems  which  were  readily  resolved  by  a  telephone  call  of  DRC.  Also  in¬ 
clude  comments  about  possible  reasons  and  solutions  for  the  problem.) 

Prompts:  Complexity;  Course  training;  Time  factora,  e.R.,  lag  between  course 
&  replication  or  time  to  finish  package;  Materiels,  Preparation  of  individual,.) 


r 


Were  the  instructions  in  the  data  packages  generally  adequate  for  Interpreting 
what  was  actually  required,  i.e.,  calculating  the  required  answers? 

If  NO,  please  explain  possible  reasons  and  stute  which  dutu  package. 


Did  you  note  any  significant  descrepancies  between  your  solutions  and  the 
"school"  (DRC)  solutions  during  the  replication? 

If  YES,  please  state  reasons,  if  known.  (Prompts:  Replicator  background; 
Training  course;  Data  package  instructions,  materials;  Other  (errors, etc.) 
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8.  What  would  have  been  required  of  you  Co  arlve  at  Che  school  solution  (DKC 
answers)?  Apply  comments  to  specific  data  packnp.es. 


9.  Did  you  read  the  pre-replication  package  materials  for  the  SINCGARS? 
If  YES,  were  they  useful? 


10.  Did  the  course  prepare  you  adequately  for  the  replication? 


EXPLAIN. . . 


11.  What  criteria  could  you  suggest  for  selecting  analyst  trainees  to  apply 
HARDMAN?  (Answer  may  vary  for  different  steps  in  the  process) 


12.  Do  you  have  any  suggestions  for  training  course  improvements? 
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This  appendix  contains  the  data  sets  and  baseline  runs  of  the  SEDCOM 
cost  model.  The  first  table  is  the  baseline  run  used  in  the  analysis.  The 
equipment  used  there  is  SINCGARS  without  the  battery.  This  was  done  because 
the  battery,  with  extremely  low  mean  time  between  failures,  takes  a  long  time 
to  run  on  the  computer  and  adds  nothing  to  the  analysis.  For  the  sake  of 
completeness  a  second  baseline  cost  estimate  is  presented,  based  on  the 
inclusion  of  the  battery.  It  is  labeled  SNCGARS2. 

Most  of  the  input  data  used  to  describe  both  SINCGARS  and  its  operating 
and  support  environments  were  provided  by  the  SINCGARS  Program  Office  and  are 
presented  in  the  next  several  tables.  Because  the  computer  program  used  for 
this  example  was  derived  from  the  Navy  SECDOM  model,  the  billet  structure  and 
some  maintenance  level  terminology  (for  example,  ship  level  is  organizational 
level)  were  merely  adopted  to  prevent  reprogramming.  The  use  of  some  Navy 
terminology  and  data  did  not  affect  the  accuracy  of  the  sensitivity  study 
because  this  effort  was  conducted  strictly  as  a  top-level  (order-of-magnitude) 
analysis . 


*  COST  ELEMENTS  ($'000) 


Life  Cycle  Cost 

885323.0 

System  Unit  Cost 

11.1 

INITIAL  COSTS: 

Production 

359265.0 

Maine.  Training 

1251.2 

Op.  Training 

80850.0 

Mlsc.  Manpower 

o.o 

Spares 

2844.2 

S+TE 

67048.6 

Tech.  Doc. 

55.2 

* ‘TOTAL 

511314.0 

OPERATION  AND  SUPPORT  COSTS: 

Malnt.  Wage 

31587.0 

Op.  +  Off.  Wage 

35684.7 

Malnt.  Training 

3842.2 

Op.  Training 

248394.0 

Mlsc.  Manpower 

0.0 

Spares 

11018.2 

Repair 

2257.9 

Transportation 

20.2 

S+TE 

41198.5 

Tech.  Doc. 

5.5 

“TOTAL 

374009.0 
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*  SYSTEM  RATES  * 


Number  of  LRA  Types  17 

Number  of  LRA' s  17 


Confidence  Against  Stockout  .999 

System  KTBF  472.3  hrs. 

System  MTTR  3.0  hrs. 


SUPPORT  POLICY  SUMMARIES: 

LRA' a  coded  COD  0 
LRA's  coded  MOD  9 
LRA's  coded  local  repair  4 
LRA's  coded  discard  4 


TRAINING  BILLETS  BY  TYPE: 
Equipment  Operators  32340 
Org  and  Int  Malnt  476 

Depot  Technicians  6 

REAL  MANNING  REQUIREMENTS: 

Org.  and  Int.  Malnt.  235.97 
Depot  Maintenance  4.92 

Operators  8085.00 


SNCCARS2 


*  COST  ELEMENTS  ($*000)  * 

******************************************* 


Life  Cycle  Cose  1126040.0 


System  Unit  Cost 

11.2 

INITIAL  COSTS: 

Production 

314545.0 

Malnt.  Training 

1618.7 

Op.  Training 

70280.0 

Misc.  Manpower 

0.0 

Spares 

54779.7 

S+TE 

67048.6 

Tech.  Doc. 

55.2 

“TOTAL 

508327.0 

OPERATION  AND  SUPPORT  COSTS: 

Malnt.  Wage 

39707.1 

Op.  +  Off.  Wage 

31019.4 

Maine.  Training 

4971.3 

Op.  Training 

215920.0 

Misc.  Manpower 

0.0 

Spares 

282912.0 

Repair 

1962.7 

Transportation 

17.6 

S+TE 

41198.5 

Tech.  Doc. 

5.5 

* ‘TOTAL 

617714.0 
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SYSTEM  RATES 


Number  of  LRA  Types  18 

Number  of  LRA's  18 


Confidence  Against  Stockout  .000 

System  MTBF  22.8  hrs. 

System  MTTR  0.4  hrs. 


SUPPORT  POLICY  SUMMARIES: 

LAA's  coded  COD  0 
LRA's  coded  MOD  9 
LRA's  coded  local  repair  4 
LRA's  coded  discard  5 


TRAINING  BILLETS  BY  TYPE: 
Equipment  Operators  28212 
Org  and  Int  Malnt  616 

Depot  Technicians  6 

REAL  MANNING  REQUIREMENTS: 

Org.  and  Int.  Malnt.  296.63 
Depot  Maintenance  4.27 

Operators  7028.00 


y  »«J  '.'•V  rj  ij;*Lil"Jf  ».l  T!  P?  P-»  ’J  ■-'  ■*  ■»  n^r.- 


File  name:  SINCGARS 


«*••*••*»«*.*•*•**••*  fJAVY  ENVIRONMENTAL  and  cost  factors  ********************* 
U  Daily  cost  of  C-school  training  (S/day/student) .  250 

1  Cost  of  A-school,  maintenance  technician  ($/student) .  0 

2  Cost  of  A-school,  operator  (S/student) .  0 

3  Annual  billet  cost,  maintenance  technician .  30583 

4  Annual  billet  cost,  operator .  23235 

5  Annual  billet  cost,  maintenance  technician  at  depot .  38018' 

6  Annual  billet  cost,  officer .  0 

7  Available  weekly  work  hours,  maintenance  technician .  48 

8  Available  weekly  work  hours,  operator .  60 

9  Available  weekly  work  hours,  depot  technician .  40 

10  Annual  turnover  rate,  shipboard  personnel  (decimal) . 5 

11  Annual  turnover  rate,  depot  personnel  (decimal).. . 33 

12  Annual  discount  rate  (decimal) . 1 

13  Condemnation  rate  (decimal) . 01 

14  S+TE  support  cost  :  S+TE  purchase  cost  (ratio) . 1 

15  Cost  of  technical  documentation  development  (S/page) .  200 

16  Annual  tech,  documentation  maintenance  rate  ($/page/yr) .  20 

17  Cost  of  insured  freight  ($/1b/mi1e) . 0002 


*************************  SYSTEM  OPERATING  ENVIRONMENT 


0  Number  of  ships  on  which  system  is  deployed .  23 

1  Number  of  systems  per  ship .  1004 

2  Length  of  system  life  cycle  (yrs) .  10 

3  Length  of  deployment  period  (days) .  5 

4  Number  of  deployments  per  year .  52 

5  Average  system  operating  hours  (lir/wk) . 15 

6  Peak  system  operating  hours  (hr/wk) .  30 

7  Reduction  rate  (learning  curve  slope) .  1 

8  Number  of  stockage  depots .  3 

9  Number  of  repair  depots .  2 

10  Cost  of  LRA  repair  at  contractor  depot .  1000 

11  Ship  response  time  (days) .  2 

12  Military  depot  response  time  (days) . 16 

13  Contractor  operated  depot  response  time  (days) .  20 

14  Distance  between  repair  and  supply  depots  (miles) .  750 

15  Other  personnel  costs  (S/person) . . .  0 


***************************  SYSTEM  design  parameters  ************************* 

1  Required  system  confidence  level  against  stockout . 95 

2  System  assembly  cost  at  given  lot  size .  5204 

3  Lot  size  for  assembly .  100 

4  Average  LRA  weight  (lb) . 1.5 

5  Fault  isolation  hardware  cost  (S/system) . 2.35714E+06 

6  Support  and  test  hardware  cost  ($/ system)... .  0 

7  Support  and  test  software  development  cost .  0 

8  Average  LRA  specific  software  development  cost .  80000 

9  Technical  documentation  for  system  description  (pages! .  3 

10  Technical  documentation  for  fault  Isolation  (pages) .  0 

11  System  scheduled  maintenance  requirement  (manhr/wk/ship). . . 25 

12  Required  number  of  operators  per  system... .  T 

13  Required  no.  officers  per  platform .  0 

14  Required  days  C-school  for  operator  training .  10 

15  Required  days  C-school  maintenance  trng  (sys  orien  +  LRA  R&R) .  10 
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LRA  Configuration  for  SINCGARS 


CHASSIS:  I 

0  Estimated  unit  cost .  1400 

1  Corresponding  lot  size . 100 

2  LRA  mean  time  between  failures  (hours) .  14006 

3  Mean  time  to  fault  isolate,  R  &  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) .  1 

5  LRA  duty  cycle  (decimal) . 1 

6  LRA  specific  support  and  test  hardware  cost . 67 

7  Training  hours  required  for  repair  of  LRA .  1 

8  LRA  repair  material  cost  per  failure .  20 

9  Pages  of  documentation  for  repair  description .  21 

lu  Quantity  of  similar  assemblies . . .  2 


CONTROL:  1 

0  Estimated  unit  cost .  375 

1  Corresponding  lot  size... .  100 

2  LRA  mean  time  between  failures  (hours) .  27855 

3  Mean  time  to  fault  isolate,  R  i  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) . . .  1 

5  LRA  duty  cycle  (decimal) . 1 

6  LRA  specific  support  and  test  hardware  cost . 67 

7  Training  hours  required  for  repair  of  LRA .  1 

8  LRA  repair  material  cost  per  failure..... .  30 

9  Pages  of  documentation  for  repair  description .  21 

10  Identity  of  similar  assemblies . . .  1 


PWR  ASSY:  1 

0  Estimated  unit  cost .  180 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) .  25974 

3  Mean  time  to  fault  Isolate,  R  i  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) . . .  1 

5  LRA  duty  cycle  (decimal) .  1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA .  1 

8  LRA  repair  material  cost  per  failure .  30 

9  Pages  of  documentation  for  repair  description .  21 

10  Quantity  of  similar  assemblies .  1 


TUNR/MXR:  1 

0  Estimated  unit  cost .  370 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) . 12937 

3  Mean  time  to  fault  Isolate,  R  &  R  this  LRA  (hours) .  3 

4  Mean  tine  to  repair  LRA  (hours) .  1 

5  LRA  duty  cycle  (decimal) . 1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA . 1 

8  LRA  repair  material  cost  per  failure.. .  30 

9  Pages  of  documentation  for  repair  description .  21 

1U  Quantity  of  similar  assemblies .  1 


LRA  Configuration  for  SliCGARS 


IF  UEMUU:  1 

0  Estimated  unit  cost .  325 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) .  18868 

3  Mean  time  to  fault  isolate,  R  4  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LKA  (hours) .  1 

5  LSA  duty  cycle  (decimal) .  1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA .  1 

8  LRA  repair  material  cost  per  failure .  30 

9  Pages  of  documentation  for  repair  description .  21 

10  Quantity  of  similar  assemblies .  1 


EXCITER:  1 

0  Estimated  unit  cost .  950 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) .  11737 

3  Mean  time  to  fault  isolate,  R  4  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) .  1 

5  LRA  duty  cycle  (decimal) .  1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA .  1 

8  LRA  repair  material  cost  per  failure .  30 

9  Pages  of  documentation  for  repair  description .  21 

10  Quantity  of  similar  assemblies .  2 


SYHTHSIZER:  1 

0  Estimated  unit  cost .  525 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) .  15674 

3  Mean  time  to  fault  isolate,  R  4  R  this  LRA  (hours) .  3 

4  Kean  time  to  repair  LRA  (hours) .  1.5 

5  LRA  duty  cycle  (decimal) . 1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA .  1 

8  LRA  repair  material  cost  per  failure .  30 

9  Pages  of  documentation  for  repair  description .  21 

10  Quantity  of  similar  assemblies . 2 


TWO-WIRE:  1 

0  Estimated  unit  cost .  160 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) .  22624 

3  Mean  time  to  fault  isolate,  R  4  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) .  1 

5  LRA  duty  cycle  (decimal) .  1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA .  0 

8  LRA  repair  material  cost  per  failure .  200 

9  Pages  of  documentation  for  repair  description .  21 

10  Quantity  of  similar  assemblies .  3 


LRA  Configuration  for  S1MCGARS 


SWITCH:  1 

0  Estimated  unit  cost .  400 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) .  12837 

3  Mean  time  to  fault  Isolate.  R  S  R  this  LRA  (hours) .  3 

4  (lean  time  to  repair  LRA  (hours) . . .  1 

5  LRA  duty  cycle  (decimal) .  1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA .  1 

8  LRA  repair  material  cost  per  failure . . . . .  30 

9  Rages  of  documentation  for  repair  description .  21 

10  Quantity  of  similar  assemblies .  2 


REMOTE  1/0:  1 

0  Estimated  unit  cost . 225 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours).. . 44444 

3  Mean  time  to  fault  isolate,  R  4  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) .  1 

5  LRA  duty  cycle  (decimal) . 1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA .  1 

8  LRA  repair  material  cost  per  failure .  30 

9  Pages  of  documentation  for  repair  description .  21 

10  Quantity  of  similar  assemblies .  1 


AHT  C0MTRL:  1 

0  Estimated  unit  cost .  175 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) .  28818 

3  Mean  time  to  fault  Isolate,  R  S  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) .  1 

5  LRA  duty  cycle  (decimal) . 1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA.. .  1 

8  LRA  repair  material  cost  per  failure .  30 

9  Pages  of  documentation  for  repair  description .  21 

10  Quantity  of  similar  assemblies .  1 


AUD  PWR  SUP:  1 

0  Estimated  unit  cost .  210 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) .  32573 

3  Mean  time  to  fault  Isolate,  R  J  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) . . . 1 

5  LRA  duty  cycle  (decimal) .  1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA . 1 

8  LRA  repair  material  cost  per  failure . 30 

9  Pages  of  documentation  for  repair  description .  21 

10  Quantity  of  similar  assemblies...., .  1 


LKA  Configuration  for  SIMCGARS 


AUD  DATA  I/O:  1 

0  Estimated  unit  cost . 260 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) .  14881 

3  Mean  time  to  fault  isolate,  R  4  R  this  LKA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) .  1 

5  LRA  duty  cycle  (decimal) .  1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA .  1 

8  LRA  repair  material  cost  per  failure .  30 

9  Pages  of  documentation  for  repair  description .  21 

10  quantity  of  similar  assemblies .  1 


BATTERY  CASE:  1 

0  Estimated  unit  cost .  50 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours) .  86957 

3  Mean  time  to  fault  isolate,  R  &  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) .  10 

5  LRA  duty  cycle  (decimal) .  1 

6  LKA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA .  0 

8  LRA  repair  material  cost  per  failure .  60 

9  Pages  of  documentation  for  repair  description .  21 

10  Quantity  of  similar  assemblies . 3 


AUU  DAT  C0NTRL:  1 

0  Estimated  unit  cost . 180 

1  Corresponding  lot  size .  100 

2  LRA  mean  time  between  failures  (hours).. . 21978 

3  Mean  time  to  fault  Isolate,  R  4  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours)....... .  1 

5  LRA  duty  cycle  (decimal) .  1 

6  LRA  specific  support  and  test  hardware  cost .  67 

7  Training  hours  required  for  repair  of  LRA .  1 

8  LRA  repair  material  cost  per  failure. .  30 

9  Pages  of  documentation  for  repair  description .  21 

10  quantity  of  similar  assemblies .  1 


ANTENNA:  1 

0  Estimated  unit  cost . 40 

1  Corresponding  lot  size .  100 

2  LKA  mean  time  between  failures  (hours) .  13158 

3  lean  time  to  fault  Isolate,  R  4  R  this  LKA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) .  2 

5  LKA  duty  cycle  (decimal) .  1 

6  LRA  specific  support  and  test  hardware  cost .  0 

7  Training  hours  required  for  repair  of  LRA .  0 

8  LRA  repair  material  cost  per  failure .  0 

9  Pages  of  documentation  for  repair  description .  0 

10  quantity  of  similar  assemblies . .  3 


LRA  Configuration  for  S I fiCGARS 


HANDSET :  1 

0  Estimated  unit  cost . . . 80 

1  Corresponding  lot  size. . 100 

2  LRA  mean  time  between  failures  (hours)..... .  780 

3  Mean  time  to  fault  isolate,  R  5  R  this  LRA  (hours) .  3 

4  Mean  time  to  repair  LRA  (hours) .  2 

5  LRA  duty  cycle  (decimal) .  1 

6  LRA  specific  support  and  test  hardware  cost . 0 

7  Training  hours  required  for  repair  of  LRA . 0 

8  LRA  repair  material  cost  per  failure . 0 

9  Pages  of  documentation  for  repair  description . 0 

10  Quantity  of  similar  assemblies . ** 


VVT“  Ivys',  yyvjr- 


CHASSIS 


INPUT  DATA 


Quantity 

1 

uc(tot) 

1400 

lot 

100  HTllF 

14006 

HTkR 

3.0 

MTTR 

1.0 

Duty  cycle 

1.0  S*TE 

67 

Trng  hrs 

1 

Repair  parts  $ 

2U 

Tech  doc  pp 

21  USA 

2 

SUPPORT  PUL1CT  SUMMARIES 


Contractor 

Oepot 

Military 

Oepot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

46846.4 

40340.8 

40227.0 

49635.4 

INITIAL  COSTS: 

Production 

39356.8 

39356.8 

39356.8 

39356.8 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

284.5 

280.3 

172.5 

1830.2 

S*T£ 

0.0 

80.1 

81.9 

0.0 

Tech.  Ooc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

39641.3 

39721.5 

39616.2 

41187.0 

OPERATION  AUO  SUPPORT  COSTS: 

Uage 

251.9 

344.9 

335.8 

251.9 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

82.0 

82.0 

82.0 

8196.5 

Kepa 1 r 

6871.3 

137.4 

137.4 

0.0 

Transportation 

0.0 

3.1 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech.  Ooc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

7205. 1 

619.3 

610.8 

8448.4 

Support  Posture  Assigned:  local  Repair 


CONTROL 


INPUT  uaTA 

Quantity 

1 

UC(lOt) 

375  lot 

100  MTBF 

278S5 

NTRR 

3.0 

MTTK 

1.0  Duty  cycle 

1.0  S*TE 

67 

Trng  hrs 

1 

Repair  parts  S 

30  Tech  doc  pp 

21  QSA 

CT  SUMMARIES  •***’ 

1 

Contractor 

Depot 

Military 

Uepot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

14134.4 

11016.6 

11001.7 

12029.9 

INITIAL  COSTS: 

Production 

Training 

Spares 

S+TE 

Tech.  Doc. 

10542.0 

0.0 

49.7 

0.0 

0.0 

10542.0 

0.1 

48.6 

80.1 

4.2 

10542.0 

0.9 

33.6 

81.9 

4.2 

10542.0 

0.0 

257.4 

0.0 

0.0 

••TOTAL 

10591.7 

10675.0 

10662.6 

10799.4 

OPERATION  AND  SUPPORT  COSTS: 
Wage 

Training 

Spares 

Repair 

Transportation 

S+TE 

Tech.  Ooc. 

126.7 

0.0 

11.0 

3455.0 

0.0 

0.0 

0.0 

173.4 

0.1 

11.0 

103.6 

1.6 

49.2 

2.6 

168.9 

2.7 

11.0 

103.6 

0.0 

50.3 

2.6 

126.7 

0.0 

1103.9 

0.0 

0.0 

0.0 

O.C 

••TOTAL 

3592.7 

341.6 

339.1 

1230.6 

Support  Posture  Assigned:  Military  Depot 


PH K  ASSY 


IWUT  DATA 

Ouantity 

1 

UC(lot) 

180  1 ot 

100  HT8F 

25974 

iiTkk 

3.0 

MTTR 

1.0  Duty  cycle 

1.0  S+TE 

67 

Trng  hrs 

l 

Repair  parts  $ 

30  Tech  doc  pp 

21  OS  A 

1 

SUPPORT  POLICY  SUMMARIES 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

8930.8 

5524.9 

5516.8 

5895.6 

INITIAL  COSTS: 

Product i on 

S060.2 

5060.2 

5060.2 

5060.2 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

23.9 

23.9 

16.2 

131.4 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Ooc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

5084.1 

5168.5 

5163.3 

5191.6 

OPERATION  AND  SUPPORT  COSTS: 

Wage 

135.8 

186.0 

181.1 

135.8 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

5.7 

5.7 

5.7 

568.3 

Repair 

3705.2 

111.2 

111.2 

0.0 

Transportation 

0.0 

1.7 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech.  Doc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

3846.7 

356.4 

353.5 

704.1 

Support  Posture  Assigned:  Military  Depot 


TUUR/HXH 


f  >IIM  *▼ 

»  >w  U  •  <*•«•'« 


(Quantity 

1 

UC(lot) 

370  lot 

100  MTBF 

12937 

iithr 

3.0 

HTTK 

1.0  Uuty  cycle 

1.0  S*TE 

67 

Trng  hrs 

1 

Repair  parts  $ 

30  Tech  doc  pp 

21  QSA 

1 

SUPPORT  POLICY  SUMMARIES 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

18213.3 

11236.7 

11200.1 

13538.3 

INITIAL  COSTS: 

Production 

10401.4 

10401.4 

10401.4 

10401.4 

Training 

0.0 

0.1 

0.9 

O.o 

Spares 

76.6 

75.5 

45.9 

519.0 

S*TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Doc. 

0.0 

4.2 

4.2 

O.C 

••TOTAL 

10473.1 

10561.4 

10534.3 

10920.4 

OPERATIC!)  AMD  SUPPORT  COSTS: 

Wage 

272.7 

373.4 

363.6 

272.7 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

23. S 

23.5 

23.5 

2345. 

Repair 

7439.0 

223.2 

223.2 

0.- 

Transportation 

0.0 

3.3 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech.  Doc. 

0.0 

2.6 

2.6 

O.i. 

••TOTAL 

7735.2 

675.3 

665.8 

2617.9 

Support  Posture  Assigned:  Military  Oepot 


IF  UEhOU 


I'irUT  u«Ta 


Quantity 

1 

UC(lot) 

325  lot 

100  htbf 

1886b 

HTRR 

3.0 

MTTK 

1.0  Duty  cycle 

1.0  StTE 

67 

Trng  hrs 

1 

Repair  parts  $ 

30  Tech  doc  pp 

21  QSA 

1 

SUPPORT  POLICY  SUMMARIES 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

14493.2 

9752.3 

9725.4 

11057.0 

IrtlTlAL  COSTS: 

Production 

9136.4 

9136.4 

9136.4 

9136.4 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

55.0 

54.0 

30.0 

321.1 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Uoc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

9191.4 

9274.8 

9253.4 

9457.5 

OPE HAT I OH  AHU  SUPPORT  COSTS: 

Rage 

187.0 

256.0 

249.3 

187.0 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

14.1 

14.1 

14.1 

1412. « 

Repair 

5100.6 

153.0 

153.0 

0.0 

Transportation 

0.0 

2.3 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

o.c 

Tech.  Ooc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

5301.7 

477.4 

472.0 

1599.4 

Support  Posture  Assigned:  Military  Depot 


EXCITER 


input  uata 


Quantity 

1 

UC(lot) 

9  SO  lot 

100  NTBF 

11737 

MTKR 

3.0 

flTTR 

1.0  Duty  cycle 

1.0  S*TE 

67 

Trng  hrs 

1 

Repair  parts  $ 

30  Tech  doc  pp 

21  QSA 

2 

SUPPORT  POLICY  SUMMARIES 


Contractor 

Uepot 

Military 

Depot 

Local 

Repair 

Discard 

LIKE  CYCLE  COST 

3S476.6 

27774.0 

27681.1 

35102.8 

INITIAL  COSTS: 

Product  1  on 

26706.4 

26706.4 

26706.4 

20706.4 

Training 

U.O 

0.1 

0.9 

0.0 

Spares 

203.6 

203.6 

119.1 

1458.7 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Doc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

26910.0 

26994.4 

26912.4 

28165.1 

OPERATION  AND  SUPPORT  COSTS: 

Uage 

300.6 

411.6 

400.8 

300.6 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

66.4 

66.4 

66.4 

6637 • 1 

Repair 

8199.6 

246.0 

246.0 

0.0 

Transportation 

0.0 

3.7 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech.  Doc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

8S66.6 

779.6 

768.7 

6937.7 

Support  Posture  Assigned:  Local  Repair 


SYNTHSIZER 


INPUT  DATA 


Quantity 

1 

UC  (lot) 

525  lot 

100 

KTBF 

15674 

I1TKR 

3.0 

HTTP 

1.5  Duty  cycle 

1.0 

S+TE 

67 

Trng  hrs 

1 

Repair  parts  $ 

30  Tech  doc  pp 

21 

QSA 

2 

SUPPORT  POLICY  SUMMARIES 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

21245.9 

1SSS2.3 

15500.0 

18342.8 

INITIAL  COSTS: 

Production 

14758.8 

14758.8 

14758.8 

14758.8 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

94.5 

92.9 

49.3 

612.3 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Doc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

14853.3 

14936.1 

14895.1 

15371.1 

OPERATION  AND  SUPPORT  COSTS: 

Wage 

225.1 

349.8 

337.6 

225.! 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

27.5 

27.5 

27.5 

2746.6 

Repair 

6140.0 

184.2 

184.2 

O.C 

Transportation 

0.0 

2.8 

0.0 

O.C 

S*TE 

0.0 

49.2 

50.3 

0.0 

Tech.  Doc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

6392.6 

616.1 

604.9 

2971.7 

Support  Posture  Assigned:  Local  Repair 


>yy 


TWO-WIRE 


I  .'(PUT  DATA 

Quantity 
iiTkh 
Trng  hrs 


1  UC(lot) 

3.0  MTTK 
0  Repair  parts  $ 


160  lot 

1.0  Duty  cycle 

200  Tech  doc  pp 


100  HTBF 
1.0  S+TE 
21  QSA 


SUPPORT  POLICY  SUU'IARIES 


Contractor 

Depot 

Military 

Depot 

Local 

Repa i r 

Discard 

FE  CYCLE  COST 

6936.4 

5728. 9 

5715.9 

5366.3 

IITIAL  COSTS: 

Production 

4497.9 

4497.9 

4497.9 

4497.9 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

22.9 

22.9 

14.5 

132.5 

S*TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Doc. 

0.0 

4.2 

4.2 

0.0 

TOTAL 

4S20.8 

460S.1 

4598.5 

4630.4 

OPERATION  AND  SUPPORT  COSTS: 


Wage 

155.9 

213.5 

207.9 

155.9 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

5.8 

5.8 

5.8 

579.9 

Repair 

4253.8 

850.8 

850.8 

0.0 

Transportation 

0.0 

1.9 

0.0 

0.0 

S*TE 

0.0 

49.2 

50.3 

0.0 

Tech.  Doc. 

0.0 

2.6 

2.6 

0.0 

•TOTAL 

4415.6 

1123.8 

1117.4 

735.9 

Support  Posture  Assigned:  Discard 


[sSV*  v  -  mr.-  •.  v  vvar-  r?w 


’  Ti  'X.’  .i  V.  r  k  "  *  *  "V  -  J 


input  uata 

Quantity 
mTnk 
Trng  hrs 


1  l)C  (lot) 

3.0  NTTR 

1  Repair  parts  $ 


SNITCH 


<00  lot 
1.0  Duty  cycle 
30  Tech  doc  pp 


100  MTBF 
1.0  S+TE 
21  QSA 


SUPPORT  POLICY  SUMMARIES 


Contractor 

Oepot 

Military 

Depot 

Local 

Repair 

Discard 

LIKE  CYCLE  COST 

19125.0 

12093.0 

12053.9 

14639.6 

INITIAL  COSTS: 

Production 

11244.8 

11244.8 

11244.8 

11244.8 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

82.9 

81.7 

49.7 

564.8 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Ooc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

11327.7 

11410.9 

11381.4 

11809.6 

KMTE  I/O 


INPUT  data 


Quantity 

1 

UC(Iot) 

225  lot 

100  tlTSF 

44444 

I1TKR 

3.0 

MTTR 

1.0  Outy  cycle 

1.0  S*TE 

67 

Trng  hrs 

1 

Repair  parts  $ 

30  Tech  doc  pp 

21  QSA 

1 

SUPPORT  POLICY  SUMMARIES 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIKE  CYCLE  COST 

8596. S 

6662.0 

6656.1 

6919.9 

Initial  costs: 

Production 

6325.2 

6325.2 

6325.2 

6325.2 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

22.4 

21.7 

13.4 

100.2 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Doc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

6347.6 

6431.3 

6425.5 

6425.4 

OPERATION  AMD  SUPPORT  COSTS: 

Uage 

79.4 

108.7 

105.8 

79.4 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

4.2 

4.2 

4.2 

415.1 

Repair 

2165.4 

65.0 

65.0 

0.0 

Transportation 

0.0 

1.0 

0.0 

0.0 

S*T£ 

0.0 

49.2 

50.3 

0.0 

Tech.  Uoc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

2248.9 

230.7 

230.5 

494.5 

Support  Posture  Assigned:  Military  Depot 


AIIT  COhTRL 


Input  data 


Quantity 

I 

UC(lot) 

175  lot 

100  htbf 

281118 

HTKR 

3.0 

MTTR 

1.0  Uuty  cycle 

1.0  S+TE 

67 

Trng  hrs 

1 

Repair  parts  S 

30  Tech  doc  pp 

21  QSA 

1 

SUPPORT  PUL  ICY  SUMMARIES 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

8409.7 

5352.9 

5346.2 

5656.8 

INITIAL  COSTS: 

Production 

4919.6 

4919.6 

4919.6 

4919.6 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

23.2 

22.7 

15.7 

116.8 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Doc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

4942.8 

5026.6 

5022.2 

5036.4 

OPERATION  ANO  SUPPORT  COSTS: 

Uage 

122.4 

167.6 

163.2 

122.4 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

5.0 

5.0 

5.0 

498.0 

Repair 

3339.5 

100.2 

100.2 

0.0 

Transportation 

0.0 

1.5 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech.  Ooc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

3466.9 

326.2 

324.0 

620.4 

Support  Posture  Assigned:  Military  Depot 


AUD  PUR  SUP 


INPUT  DATA 


(foantity 

1 

UC(lot) 

210  lot 

100  HTBF 

32573 

hTKR 

3.0 

HTTR 

1.0  Duty  cycle 

1.0  S+TE 

67 

Trng  hrs 

1 

Repair  parts  $ 

30  Tech  doc  pp 

21  QSA 

1 

SUPPORT  POLICY  SUMMARIES 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

D1 scard 

LIFE  CYCLE  COST 

8998. 7 

6309.8 

6297.2 

6666.9 

UITIAL  COSTS: 

Production 

5903.5 

5903.5 

5903.5 

5903.5 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

27.0 

26.4 

12.8 

126.4 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Ooc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

5930.6 

6014.3 

6003.2 

6029.9 

OPERATION  ANO  SUPPORT  COSTS: 

Wage 

108.3 

148.3 

144.4 

108.3 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

5.3 

5.3 

5.3 

528.7 

Repair 

2954.6 

88.6 

88.6 

0.0 

Transportation 

0.0 

1.3 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech.  Doc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

3068.1 

295.5 

293.9 

637.0 

Support  Posture  Assigned:  Military  Depot 


K-2  i 


AUO  DATA  I/O 


INPUT  DATA 


Ouantity 

1 

UC(lot) 

260  lot 

100  HTBF 

14K81 

hTRA 

3.0 

NTTR 

1.0  Uuty  cycle 

1.0  S+TE 

67 

Trng  hrs 

1 

Repair  parts  $ 

30  Tech  doc  pp 

21  USA 

1 

Support  policy  sum-iaries 


Contractor 

Oepot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

14080.4 

8033.2 

8000.7 

9301.7 

INITIAL  COSTS: 

Production 

7309.1 

7309.1 

7309.1 

7309.1 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

52. 7 

51.9 

24.6 

322.8 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Doc. 

0.0 

4.2 

4.2 

0.0 

“TOTAL 

7361.8 

7445.4 

7420.6 

7631.9 

OPERATION  ANU  SUPPORT  COSTS: 

Wage 

237.1 

324.6 

316.1 

237.1 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

14.3 

14.3 

14.3 

1432.7 

Repair 

6467.2 

194.0 

194.0 

0.0 

Transportation 

0.0 

2.9 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech.  Doc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

6718.6 

587.8 

5 80.0 

1669.8 

Support  Posture  Assigned:  Military  Depot 


BATTERY  CASE 


INPUT  DATA 


Quantity 

1 

UC  (lot) 

50 

lot 

100  MTBF 

8695? 

HTKR 

3.0 

KTTR 

10.0 

Duty  cycle 

1.0  S*TE 

67 

Trng  hrs 

0 

Repair  parts  S 

60 

Tech  doc  pp 

21  QSA 

3 

SUPPORT  POLICY  SUMMARIES 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

2556.6 

1802.7 

1790.1 

1505.4 

INITIAL  COSTS: 

Production 

1405.6 

1405.6 

1405.6 

1405.6 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

3.2 

3.2 

2.9 

12.1 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Ooc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

1408.8 

1493.1 

1494.6 

1417.7 

OPERATION  AHU  SUPPORT  COSTS: 

Uage 

40.6 

190.4 

175. R 

40.6 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

0.5 

0.5 

0.5 

47.1 

Repair 

1106.7 

66.4 

66.4 

0.0 

Transportation 

0.0 

0.5 

0.0 

0.0 

S+TE 

0.0 

49.2 

50.3 

0.0 

Tech.  Ooc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

1147.8 

309.6 

295.6 

87.7 

Support  Posture  Assigned:  Oiscard 


K  -2  5 


AUD  OAT  CUWTHL 


IfiHUT  OATA 


Quantity 

1 

UC{ lot) 

100  1 ot 

100  MT8F 

21970 

MTKK 

3.0 

NTTK 

1.0  Duty  cycle 

1.0  S+TE 

67 

Trng  hrs 

1 

Repair  parts  $ 

30  Tech  do c  pp 

21  QSA 

1 

SUPPORT  POLICY  SUMMARIES 


Contractor 

Depot 

Military 

Oepot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

9635. A 

5585.5 

5571.2 

6048.4 

INITIAL  COSTS: 

Production 

5060.2 

5060.2 

5060.2 

5060.2 

Training 

0.0 

0.1 

0.9 

0.0 

Spares 

29.2 

29.2 

16.4 

156.2 

S+TE 

0.0 

80.1 

81.9 

0.0 

Tech.  Uoc. 

0.0 

4.2 

4.2 

0.0 

••TOTAL 

5089.3 

5173.7 

5163.5 

5216.3 

OPERATION  AND  SUPPORT  COSTS: 

Wage 

160.5 

219.8 

214.0 

160.5 

Training 

0.0 

0.1 

2.7 

0.0 

Spares 

6.7 

6.7 

6.7 

671.6 

Repair 

4378.9 

131.4 

131.4 

0.0 

Transportation 

0.0 

2.0 

0.0 

0.0 

S*TE 

0.0 

49.2 

50.3 

O.C 

Tech.  Doc. 

0.0 

2.6 

2.6 

0.0 

••TOTAL 

4546.1 

411.8 

407.7 

832.1 

Support  Posture  Assigned:  Military  Depot 


antenna 


INPUT  DATA 


Quantity 

1 

UC(lot) 

40 

lot 

100  HTBF 

13158 

HTKR 

3.0 

HTTR 

2.0 

Duty  cycle 

1.0  S+TE 

II 

Trng  hrs 

0 

Repair  parts  $ 

0 

Tech  doc  pp 

0  QSA 

3 

SUPPORT  POLICY  SUMMARIES 


Contractor 

Depot 

Military 

Depot 

Local 

Repair 

Oiscard 

LIFE  CYCLE  COST 

8717.5 

1733.7 

17U7.9 

1697.2 

INITIAL  COSTS: 

Proauction 

1124.5 

1124.5 

1124.5 

1124.5 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

8.3 

8.2 

5.0 

55.3 

S+TE 

0.0 

80.0 

80.0 

0.0 

Tech.  Ooc. 

0.0 

0.0 

0.0 

0.0 

••TOTAL 

1132.8 

1212.6 

1209.4 

1179.8 

OPERATION  AflO  SOPPORT  COSTS: 

Uage 

263.1 

466.1 

446.9 

268.1 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

2.5 

2.5 

2.5 

249.3 

Repair 

7314.1 

0.0 

0.0 

0.0 

Transportation 

0.0 

3.3 

0.0 

0.0 

S*TE 

0.0 

49.2 

49.2 

0.0 

Tech.  Ooc. 

0.0 

0.0 

0.0 

0.0 

••TOTAL 

7584.7 

521.1 

498.5 

517.4 

Support  Posture  Assigned:  Discard 


handset 


I ftPUT  JATA 

quantity  1  UC(iot)  80  lot 

hTkK  3.0  MTTR  2.0  Duty  cycle 

Trntj  hrs  0  Repair  parts  J  0  Tech  doc  pp 


100  MTBF 
1.0  S+TE 
0  USA 


730 

0 

3 


Contractor 

Oepot 

LIFE  CYCLE  COST 

130370.0 

INITIAL  COSTS: 

Production 

2249.0 

Training 

0.0 

Spares 

130.6 

S*TE 

0.0 

Tech.  Ooc. 

0.0 

••TOTAL 

2379.6 

SUPPORT  POLICY  SUMMARIES 


Military 

Local 

Oepot 

Repair 

Oiscard 

10611.9 

10063.6 

16903.1 

2249.0 

2249.0 

2249.0 

0.0 

0.0 

0.0 

130.6 

63.1 

1721.0 

80.0 

80.0 

0.0 

0.0 

0.0 

0.0 

2469.6 

2392.1 

3969.9 

OPERATION  ANU  SUPPORT  COSTS: 
Uage 

Training 

Spares 

Repair 

Transportation 

S+TE 

Tech.  Doc. 

••TOTAL 


4522.9 

7863.5 

0.0 

0.0 

84.1 

84.1 

123383.0 

0.0 

0.0 

55.5 

0.0 

49.2 

0.0 

0.0 

127990.0 

8052.3 

7538.2 

4522.9 

0.0 

0.0 

84.1 

3410.3 

0.0 

0.0 

0.0 

0.0 

49.2 

0.0 

U.O 

0.0 

7671.5 

12933.2 

Support  Posture  Assigned:  Oiscard 


I. 


i 


r.'« . 
r  v. 


■  M 


E-28 


BATTERY 


INPUT  DATA 


Quantity 
fiTRK 
Trng  hr* 


1  UC(lot) 

0.3  flTTR 
0  Repair  parts  S 


80  lot 

2.0  Duty  cycle 
0  Tech  doc  pp 


100  MTBF 
1.0  S+TE 
0  QSA 


24 

0 

3 


SUPPORT  POLICY  SUMMARIES 


Contractor 

Oepot 

Military 

depot 

Local 

Repair 

Discard 

LIFE  CYCLE  COST 

X4030330.il 

130887.0 

116801.0 

342672.0 

INITIAL  COSTS: 

Production 

2249.0 

2249.0 

2249.0 

2249.0 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

31S3.S 

31S3.5 

1442.7 

54839.4 

S+TE 

U.O 

80.0 

80.0 

0.0 

Tech.  Doc. 

0.0 

0.0 

0.0 

0.0 

••TOTAL 

S402.S 

5482. S 

3771.7 

57088.4 

OPERATION  AMD  SUPPORT  COSTS: 

Uage 

12249.6 

120818.0 

110247.0 

12249.6 

Training 

0.0 

0.0 

0.0 

0.0 

Spares 

2733.3 

2733.3 

2733.3 

273334.0 

Repair 

X4 009950.0 

0.0 

0.0 

0.0 

Transportation 

0.0 

1804.5 

0.0 

0.0 

S*TE 

0.0 

49.2 

49.2 

0.0 

Tech.  Doc. 

0.0 

0.0 

0.0 

0.0 

••TOTAL 

X4024930.0 

125405.0 

113029.0 

285584.0 

Support  Posture  Assigned:  Discard 


122785 


E-29 


